Modules

This section describes the functionality of the various Python modules that make up PSyclone.

Module: f2pygen

f2pygen provides functionality for generating Fortran code from scratch and supports the addition of a use statement to an existing parse tree.

Variable Declarations

Three different classes are provided to support the creation of variable declarations (for intrinsic, character and derived-type variables). An example of their use might be:

>>> from psyclone.f2pygen import (ModuleGen, SubroutineGen, DeclGen,
... CharDeclGen, TypeDeclGen)
>>> module = ModuleGen(name="testmodule")
>>> sub = SubroutineGen(module, name="testsubroutine")
>>> module.add(sub)
>>> sub.add(DeclGen(sub, datatype="integer", entity_decls=["my_int"]))
>>> sub.add(CharDeclGen(sub, length="10", entity_decls=["my_char"]))
>>> sub.add(TypeDeclGen(sub, datatype="field_type", entity_decls=["ufld"]))
>>> gen = str(module.root)
>>> print(gen)
  MODULE testmodule
    IMPLICIT NONE
    CONTAINS
    SUBROUTINE testsubroutine()
      TYPE(field_type) ufld
      CHARACTER(LEN=10) my_char
      INTEGER my_int
    END SUBROUTINE testsubroutine
  END MODULE testmodule

The full interface to each of these classes is detailed below:

class psyclone.f2pygen.DeclGen(parent, datatype='', entity_decls=None, intent='', pointer=False, kind='', dimension='', allocatable=False, save=False, target=False, initial_values=None, private=False)[source]

Generates a Fortran declaration for variables of various intrinsic types (integer, real and logical). For character variables CharDeclGen should be used.

Parameters

  • parent (psyclone.f2pygen.BaseGen) – node to which to add this declaration as a child.

  • datatype (str) – the (intrinsic) type for this declaration.

  • entity_decls (list) – list of variable names to declare.

  • intent (str) – the INTENT attribute of this declaration.

  • pointer (bool) – whether or not this is a pointer declaration.

  • kind (str) – the KIND attribute to use for this declaration.

  • dimension (str) – the DIMENSION specifier (i.e. the xx in DIMENSION(xx)).

  • allocatable (bool) – whether this declaration is for an ALLOCATABLE quantity.

  • save (bool) – whether this declaration has the SAVE attribute.

  • target (bool) – whether this declaration has the TARGET attribute.

  • initial_values (list of str with same no. of elements as entity_decls) – initial value to give each variable.

  • private (bool) – whether this declaration has the PRIVATE attribute (default is False).

Raises

RuntimeError – if datatype is not one of DeclGen.SUPPORTED_TYPES.

class psyclone.f2pygen.CharDeclGen(parent, entity_decls=None, intent='', pointer=False, kind='', dimension='', allocatable=False, save=False, target=False, length='', initial_values=None, private=False)[source]

Generates a Fortran declaration for character variables.

Parameters

  • parent (psyclone.f2pygen.BaseGen.) – node to which to add this declaration as a child.

  • entity_decls (list) – list of variable names to declare.

  • intent (str) – the INTENT attribute of this declaration.

  • pointer (bool) – whether or not this is a pointer declaration.

  • kind (str) – the KIND attribute to use for this declaration.

  • dimension (str) – the DIMENSION specifier (i.e. the xx in DIMENSION(xx)).

  • allocatable (bool) – whether this declaration is for an ALLOCATABLE quantity.

  • save (bool) – whether this declaration has the SAVE attribute.

  • target (bool) – whether this declaration has the TARGET attribute.

  • length (str) – expression to use for the (len=xx) selector.

  • initial_values (list of str with same no. of elements as entity_decls) – list of initial values, one for each variable. Each of these can be either a variable name or a literal, quoted string (e.g. “‘hello’”). Default is None.

  • private (bool) – whether this declaration has the PRIVATE attribute.

class psyclone.f2pygen.TypeDeclGen(parent, datatype='', entity_decls=None, intent='', pointer=False, dimension='', allocatable=False, save=False, target=False, is_class=False, private=False)[source]

Generates a Fortran declaration for variables of a derived type.

Parameters

  • parent (psyclone.f2pygen.BaseGen) – node to which to add this declaration as a child.

  • datatype (str) – the type for this declaration.

  • entity_decls (list) – list of variable names to declare.

  • intent (str) – the INTENT attribute of this declaration.

  • pointer (bool) – whether or not this is a pointer declaration.

  • dimension (str) – the DIMENSION specifier (i.e. the xx in DIMENSION(xx)).

  • allocatable (bool) – whether this declaration is for an ALLOCATABLE quantity.

  • save (bool) – whether this declaration has the SAVE attribute.

  • target (bool) – whether this declaration has the TARGET attribute.

  • is_class (bool) – whether this is a class rather than type declaration.

  • private (bool) – whether or not this declaration has the PRIVATE attribute. (Defaults to False.)

Adding code

f2pygen supports the addition of use statements to an existing fparser1 parse tree:

psyclone.f2pygen.adduse(name, parent, only=False, funcnames=None)[source]

Adds a use statement with the specified name to the supplied object. This routine is required when modifying an existing AST (e.g. when modifying a kernel). The classes are used when creating an AST from scratch (for the PSy layer).

Parameters

  • name (str) – name of module to USE

  • parent (fparser.one.block_statements.*) – node in fparser1 AST to which to add this USE as a child

  • only (bool) – whether this USE has an “ONLY” clause

  • funcnames (list) – list of quantities to follow the “ONLY” clause

Returns

an fparser1 Use object

Return type

fparser.one.block_statements.Use

The PSyclone code where the adduse function was used has recently been migrated from using fparser1 to using fparser2. In recognition of this change a new version of adduse has been developed which adds use statements to an existing fparser2 parse tree. For the timebeing this new version is located in the same file it is used - alg_gen.py - but will be migrated to f2pygen (or equivalent) in the future:

psyclone.alg_gen.adduse(location, name, only=None, funcnames=None)[source]

Add a Fortran ‘use’ statement to an existing fparser2 parse tree. This will be added at the first valid location before the current location.

This function should be part of the fparser2 replacement for f2pygen (which uses fparser1) but is kept here until this is developed, see issue #240.

Parameters

  • location (fparser.two.utils.Base) – the current location (node) in the parse tree to which to add a USE.

  • name (str) – the name of the use statement.

  • only (bool) – whether to include the ‘only’ clause in the use statement or not. Defaults to None which will result in only being added if funcnames has content and not being added otherwise.

  • funcnames (list of str) – a list of names to include in the use statement’s only list. If the list is empty or None then nothing is added. Defaults to None.

Raises

  • GenerationError – if no suitable enclosing program unit is found for the provided location.

  • NotImplementedError – if the type of parent node is not supported.

  • InternalError – if the parent node does not have the expected structure.

Module: configuration

PSyclone uses the Python ConfigParser class (https://docs.python.org/3/library/configparser.html) for reading the configuration file. This is managed by the psyclone.configuration module which provides a Config class. This class is a singleton, which can be (created and) accessed using Config.get(). Only one such instance will ever exist:

class psyclone.configuration.Config[source]

Handles all configuration management. It is implemented as a singleton using a class _instance variable and a get() function.

property api

Getter for the API selected by the user.

Returns

The name of the selected API.

Return type

str

api_conf(api=None)[source]

Getter for the object holding API-specific configuration options.

Parameters

api (str) – Optional, the API for which configuration details are required. If none is specified, returns the config for the default API.

Returns

object containing API-specific configuration

Return type

One of psyclone.configuration.LFRicConfig, psyclone.configuration.GOceanConfig or None.

Raises

  • ConfigurationError – if api is not in the list of supported APIs.

  • ConfigurationError – if the config file did not contain a section for the requested API.

property default_api

Getter for the default API used by PSyclone.

Returns

default PSyclone API

Return type

str

property default_stub_api

Getter for the default API used by the stub generator.

Returns

default API for the stub generator

Return type

str

property distributed_memory

Getter for whether or not distributed memory is enabled

Returns

True if DM is enabled, False otherwise

Return type

bool

property filename

Getter for the full path and name of the configuration file used to initialise this configuration object.

Returns

full path and name of configuration file

Return type

str

static find_file()[source]

Static method that searches various locations for a configuration file. If the full path to an existing file has been provided in the PSYCLONE_CONFIG environment variable then that is returned. Otherwise, we search the following locations, in order:

  • ${PWD}/.psyclone/

  • if inside-a-virtual-environment:

    <base-dir-of-virtual-env>/share/psyclone/

  • ${HOME}/.local/share/psyclone/

  • <system-install-prefix>/share/psyclone/

Returns

the fully-qualified path to the configuration file

Return type

str

Raises

ConfigurationError – if no config file is found

static get(do_not_load_file=False)[source]

Static function that if necessary creates and returns the singleton config instance.

Parameters

do_not_load_file (bool) – If set it will not load the default config file. This is used when handling the command line so that the user can specify the file to load.

get_constants()[source]
Returns

the constants instance of the current API.

Return type

either psyclone.domain.lfric.LFRicConstants, psyclone.domain.gocean.GOceanConstants, or psyclone.domain.nemo.NemoConstants

get_default_keys()[source]

Returns all keys from the default section. :returns list: List of all keys of the default section as strings.

static get_repository_config_file()[source]

This function returns the absolute path to the config file included in the PSyclone repository. It is used by the testing framework to make sure all tests get the same config file (see tests/config_tests for the only exception). :return str: Absolute path to the config file included in the PSyclone repository.

property include_paths
Returns

the list of paths to search for Fortran include files.

Return type

list of str.

property kernel_naming
Returns

what naming scheme to use when writing transformed kernels to file.

Return type

str

property kernel_output_dir
Returns

the directory to which to write transformed kernels.

Return type

str

load(config_file=None)[source]

Loads a configuration file.

Parameters

config_file (str) – Override default configuration file to read.

Raises

ConfigurationError – if there are errors or inconsistencies in the specified config file.

property ocl_devices_per_node
Returns

The number of OpenCL devices per node.

Return type

int

property psyir_root_name

Getter for the root name to use when creating PSyIR names.

Returns

the PSyIR root name.

Return type

str

property reprod_pad_size

Getter for the amount of padding to use for the array required for reproducible OpenMP reductions

Returns

padding size (no. of array elements)

Return type

int

property reproducible_reductions

Getter for whether reproducible reductions are enabled.

Returns

True if reproducible reductions are enabled, False otherwise.

Return type

bool

property supported_apis

Getter for the list of APIs supported by PSyclone.

Returns

list of supported APIs

Return type

list of str

property supported_stub_apis

Getter for the list of APIs supported by the stub generator.

Returns

list of supported APIs.

Return type

list of str

property valid_psy_data_prefixes
Returns

The list of all valid class prefixes.

Return type

list of str

The Config class is responsible for finding the configuration file (if no filename is passed to the constructor), parsing it and then storing the various configuration options. If PSyclone is started via pytest, the environment variable PSYCLONE_CONFIG is set to <PSYCLONEHOME/config>. This will guarantee that all tests use the config file provided in the PSyclone repository, and not a (potentially modified) user installed version.

The Config class also stores the list of supported APIs (Config._supported_api_list) and the default API to use if none is specified in either a config file or the command line (Config._default_api). Additionally, it performs some basic consistency checks on the values it obtains from the configuration file.

Since the PSyclone API to use can be read from the configuration file, it is not possible to have API-specifc sub-classes of Config as we don’t know which API is in use before we read the file. However, the configuration file can contain API-specific settings. These are placed in separate sections, named for the API to which they apply, e.g.:

[dynamo0.3]
COMPUTE_ANNEXED_DOFS = false

Having parsed and stored the options from the default section of the configuration file, the Config constructor then creates a dictionary using the list of supported APIs to provide the keys. The configuration file is then checked for API-specific sections (again using the API names from the default section) and, if any are found, an API-specifc sub-class is created using the parsed entries from the corresponding section. The resulting object is stored in the dictionary under the appropriate key. The API-specific values may then be accessed as, e.g.:

Config.get().api_conf("dynamo0.3").compute_annexed_dofs

The API-specific sub-classes exist to provide validation/type-checking and encapsulation for API-specific options. They do not sub-class Config directly but store a reference back to the Config object to which they belong.

Constants Objects

Each API provides a specific object that stores required constants. Most of these constants are hard-coded in the object, but some are taken from a section of the configuration file. The constants are provided as class variables, but an instance of it needs to be created (at least once) in order to make sure all class variables are initialised. It is therefore recommended to always use an instance of the corresponding constant class to access these constants. The constant objects make sure that this initialisation only happens the very first time - creating an instance is therefore very cheap.

There three constant objects can be imported as follows:

  • from psyclone.domain.gocean import GOceanConstants

  • from psyclone.domain.lfric import LFRicConstants

  • from psyclone.domain.nemo import NemoConstants

These objects can be used in two different ways:

  1. If the API is known, e.g. because the constant is used in an API-specific file, an instance can simply be created and used, e.g.:

    from psyclone.domain.lfric import LFRicConstants

const = LFRicConstants()

if var is in const.VALID_LOOP_BOUNDS_NAMES:
    ...

    This usage pattern can be seen in many API-specific files.

  2. In some cases a value of an API-specific constant is required in a generic function. In this case the API-specific constant object can be accessed using the config file as follows:

    from psyclone.configuration import Config

const = Config.get().api_conf().get_constants()

if some_variable is in const.VALID_INTRINSIC_TYPES:
    ...

    This pattern is used in some functions that can be called with different APIs. The following constants are used this way:

    • VALID_ARG_TYPE_NAMES

    • VALID_INTRINSIC_TYPES

    • VALID_SCALAR_NAMES

    • VALID_LOOP_TYPES

    These are the only variables that are defined across all constant objects.

Module: transformations

As one might expect, the transformations module holds the various transformation classes that may be used to modify the Schedule of an Invoke and/or the kernels called from within it.

Note

The directory layout of PSyclone is currently being restructured. As a result of this some transformations are already in the new locations, while others have not been moved yet.

The base class for any transformation must be the class Transformation:

class psyclone.psyGen.Transformation(writer=None)[source]

Abstract baseclass for a transformation. Uses the abc module so it can not be instantiated.

Parameters

writer (psyclone.psyir.backend.visitor.PSyIRVisitor) – optional argument to set the type of writer to provide to a transformation for use when constructing error messages. Defaults to FortranWriter().

abstract apply(node, options=None)[source]

Abstract method that applies the transformation. This function must be implemented by each transform. As a minimum each apply function must take a node to which the transform is applied, and a dictionary of additional options, which will also be passed on to the validate functions. This dictionary is used to provide optional parameters, and also to modify the behaviour of validation of transformations: for example, if the user knows that a transformation can correctly be applied in a specific case, but the more generic code validation would not allow this. Validation functions should check for a key in the options dictionary to disable certain tests. Those keys will be documented in each apply() and validate() function.

Note that some apply() functions might take a slightly different set of parameters.

Parameters

  • node (depends on actual transformation) – The node (or list of nodes) for the transformation - specific to the actual transform used.

  • options (dictionary of string:values or None) – a dictionary with options for transformations.

property name
Returns

the transformation’s class name.

Return type

str

validate(node, options=None)[source]

Method that validates that the input data is correct. It will raise exceptions if the input data is incorrect. This function needs to be implemented by each transformation.

The validate function can be called by the user independent of the apply() function, but it will automatically be executed as part of an apply() call.

As minimum each validate function must take a node to which the transform is applied and a dictionary of additional options. This dictionary is used to provide optional parameters and also to modify the behaviour of validation: for example, if the user knows that a transformation can correctly be applied in a specific case but the more generic code validation would not allow this. Validation functions should check for particular keys in the options dict in order to disable certain tests. Those keys will be documented in each apply() and validate() function as ‘options[“option-name”]’.

Note that some validate functions might take a slightly different set of parameters.

Parameters

  • node (depends on actual transformation) – The node (or list of nodes) for the transformation - specific to the actual transform used.

  • options (dictionary of string:values or None) – a dictionary with options for transformations.

Those transformations that work on a region of code (e.g. enclosing multiple kernel calls within an OpenMP region) must sub-class the RegionTrans class:

class psyclone.psyir.transformations.RegionTrans(writer=None)[source]

This abstract class is a base class for all transformations that act on a list of nodes. It gives access to a validate function that makes sure that the nodes in the list are in the same order as in the original AST, no node is duplicated, and that all nodes have the same parent. We also check that all nodes to be enclosed are valid for this transformation - this requires that the sub-class populate the excluded_node_types tuple.

get_node_list(nodes)[source]

This is a helper function for region based transformations. The parameter for any of those transformations is either a single node, a schedule, or a list of nodes. This function converts this into a list of nodes according to the parameter type. This function will always return a copy, to avoid issues e.g. if a child list of a node should be provided, and a transformation changes the order in this list (which would then also change the order of the nodes in the tree).

Parameters

nodes (psyclone.psyir.nodes.Node or psyclone.psyir.nodes.Schedule or a list of             :py:obj:`psyclone.psyir.nodes.Node) – can be a single node, a schedule or a list of nodes.

Returns

a list of nodes.

Return type

list of psyclone.psyir.nodes.Node

Raises

TransformationError – if the supplied parameter is neither a single Node, nor a Schedule, nor a list of Nodes.

validate(nodes, options=None)[source]

Checks that the nodes in node_list are valid for a region transformation.

Parameters

  • nodes (subclass of psyclone.psyir.nodes.Node or a list of subclasses of psyclone.psyir.nodes.Node) – list of PSyIR nodes or a single node.

  • options (dictionary of string:values or None) – a dictionary with options for transformations.

  • options["node-type-check"] (bool) – this flag controls if the type of the nodes enclosed in the region should be tested to avoid using unsupported nodes inside a region.

Raises

  • TransformationError – if the nodes in the list are not in the original order in which they are in the AST, a node is duplicated or the nodes have different parents.

  • TransformationError – if any of the nodes to be enclosed in the region are of an unsupported type.

  • TransformationError – if the parent of the supplied Nodes is not a Schedule or a Directive.

  • TransformationError – if the nodes are in a NEMO Schedule and the transformation acts on the child of a single-line If or Where statment.

  • TransformationError – if the supplied options are not a dictionary.

Finally, those transformations that act on a Kernel must sub-class the KernelTrans class:

In all cases, the apply method of any sub-class must ensure that the validate method of the parent class is called.

Module: psyGen

Provides the base classes for PSy-layer code generation.

Module: dynamo0p3

Specialises various classes from the psyclone.psyGen module in order to support the Dynamo 0.3 API.

When constructing the Fortran subroutine for either an Invoke or Kernel stub (see Kernel-stub Generator), there are various groups of related quantities for which variables must be declared and (for Invokes) initialised. Each of these groupings is managed by a distinct sub-class of the DynCollection abstract class:

(A single base class is used for both Invokes and Kernel stubs since it allows the code dealing with variable declarations to be shared.) A concrete sub-class of DynCollection must provide an implementation of the _invoke_declarations method. If the quantities associated with the collection require initialisation within the PSy layer then the initialise method must also be implemented. If stub-generation is to be supported for kernels that make use of the collection type then an implementation must also be provided for _stub_declarations.

Although instances of (sub-classes of) DynCollection handle all declarations and initialisation, there remains the problem of constructing the list of arguments for a kernel (or kernel stub). The psyclone.dynamo0p3.ArgOrdering base class provides support for this:

This class is then sub-classed in order to support the generation of argument lists when calling kernels (KernCallArgList) and when creating kernel stubs (KernStubArgList). KernCallArgList is only used in DynKernelArguments.raw_arg_list(). KernStubArgList is only used in DynKern.gen_stub(). These classes make use of DynCollection sub-classes in order to ensure that argument naming is consistent.