OpenHMPP HMPP1 for Hybrid Multicore Parallel Programming - programming standard for Heterogeneous computing Based on a set of compiler directives, standard is a programming model designed to handle hardware accelerators without the complexity associated with GPU programming This approach based on directives has been implemented because they enable a loose relationship between an application code and the use of a hardware accelerator HWA


  • 1 Introduction
  • 2 OpenHMPP concept
    • 21 The OpenHMPP codelet concept
    • 22 Codelet RPCs
    • 23 HMPP Memory Model
    • 24 Directives concept
    • 25 Concept of set of directives
    • 26 OpenHMPP Directives Syntax
      • 261 General syntax
      • 262 Directive parameters
  • 3 OpenHMPP directives
    • 31 Directives for declaring and executing a codelet
    • 32 Data transfers directives to optimize communication overhead
    • 33 Sharing data between codelets
    • 34 Global variable
    • 35 Acceleration of regions
  • 4 Implementations
  • 5 See also
  • 6 References
  • 7 External links


The OpenHMPP directive-based programming model offers a syntax to offload computations on hardware accelerators and to optimize data movement to/from the hardware memory

The model is based on works initialized by CAPS Compiler and Architecture for Embedded and Superscalar Processors, a common project from INRIA, CNRS, the University of Rennes 1 and the INSA of Rennes

OpenHMPP conceptedit

OpenHMPP is based on the concept of codelets, functions that can be remotely executed on HWAs

The OpenHMPP codelet conceptedit

A codelet has the following properties:

  1. It is a pure function
    • It does not contain static or volatile variable declarations nor refer to any global variables except if these have been declared by a HMPP directive “resident”
    • It does not contain any function calls with an invisible body that cannot be inlined This includes the use of libraries and system functions such as malloc, printf,
    • Every function call must refer to a static pure function no function pointers
  2. It does not return any value void function in C or a subroutine in Fortran
  3. The number of arguments should be fixed ie it can not be a variadic function as in stdargh in C
  4. It is not recursive
  5. Its parameters are assumed to be non-aliased see Aliasing computing and Pointer aliasing
  6. It does not contain callsite directives ie RPC to another codelet or other HMPP directives

These properties ensure that a codelet RPC can be remotely executed by a HWA This RPC and its associated data transfers can be asynchronous

Codelet RPCsedit

HMPP provides synchronous and asynchronous RPC Implementation of asynchronous operation is hardware dependent

Synchronous versus asynchronous RPC

HMPP Memory Modeledit

HMPP considers two address spaces: the host processor one and the HWA memory

HMPPP memory Model

Directives conceptedit

The OpenHMPP directives may be seen as “meta-information” added in the application source code They are safe meta-information ie they do not change the original code behavior They address the remote execution RPC of a function as well as the transfers of data to/from the HWA memory

The table below introduces the OpenHMPP directives OpenHMPP directives address different needs: some of them are dedicated to declarations and others are dedicated to the management of the execution

Control flow instructions Directives for data management
Declarations codelet
Operational Directives callsite

Concept of set of directivesedit

One of the fundamental points of the HMPP approach is the concept of directives and their associated labels which makes it possible to expose a coherent structure on a whole set of directives disseminated in an application

There are two kinds of labels:

  • One associated to a codelet In general, the directives carrying this kind of labels are limited to the management of only one codelet called stand-alone codelet in the remainder of the document to distinguish it from the group of codelets
  • One associated to a group of codelets These labels are noted as follow: “<LabelOfGroup>“, where “LabelOfGroup” is a name specified by the user In general, the directives which have a label of this type relate to the whole group The concept of group is reserved to a class of problems which requires a specific management of the data throughout the application to obtain performance

OpenHMPP Directives Syntaxedit

In order to simplify the notations, regular expressions will be used to describe the syntax of the HMPP directives

The color convention below is used for the description of syntax directives:

  • Reserved HMPP keywords are in green;
  • Elements of grammar which can be declined in HMPP keywords are in red;
  • User’s variables remain in black

General syntaxedit

The general syntax of OpenHMPP directives is:

  • For C language:
#pragma hmpp <grp_label> codelet_label directive_type ,directive_parameters &
  • For FORTRAN language:
!$hmpp <grp_label> codelet_label directive_type ,directive_parameters &


  • <grp_label>: is a unique identifier naming a group of codelets In cases where no groups are defined in the application, this label can simply miss Legal label name must follow this grammar: a-z,A-Z,_a-z,A-Z,0-9,_ Note that the “< >” characters belong to the syntax and are mandatory for this kind of label
  • codelet_label: is a unique identifier naming a codelet Legal label name must follow this grammar: a-z,A-Z,_a-z,A-Z,0-9,_
  • directive: is the name of the directive;
  • directive_parameters: designates some parameters associated to the directive These parameters may be of different kinds and specify either some arguments given to the directive either a mode of execution asynchronous versus synchronous for example;
  • &: is a character used to continue the directive on the next line same for C and FORTRAN

Directive parametersedit

The parameters associated to a directive may be of different types Below are the directive parameters defined in OpenHMPP:

  • version = majorminormicro: specifies the version of the HMPP directives to be considered by the preprocessor
  • argsarg_itemssize=: specifies the size of a non scalar parameter an array
  • argsarg_itemsio=in|out|inout: indicates that the specified function arguments are either input, output or both By default, unqualified arguments are inputs
  • cond = "expr": specifies an execution condition as a boolean C or Fortran expression that needs to be true in order to start the execution of the group or codelets
  • target=target_name:target_name: specifies which targets to try to use in the given order
  • asynchronous: specifies that the codelet execution is not blocking default is synchronous
  • args<arg_items>advancedload=true: indicates that the specified parameters are preloaded Only in or inout parameters can be preloaded
  • argsarg_itemsnoupdate=true: this property specifies that the data is already available on the HWA and so that no transfer is needed When this property is set, no transfer is done on the considered argument
  • args<arg_items>addr="<expr>": <expr> is an expression that gives the address of the data to upload
  • args<arg_items>const=true: indicates that the argument is to be uploaded only once

OpenHMPP directivesedit

Directives for declaring and executing a codeletedit

A codelet directive declares a computation to be remotely executed on a hardware accelerator For the codelet directive:

  • The codelet label is mandatory and must be unique in the application
  • The group label is not required if no group is defined
  • The codelet directive is inserted just before the function declaration

The syntax of the directive is:

#pragma hmpp <grp_label> codelet_label codelet , version = majorminormicro , argsarg_itemsio=in|out|inout , argsarg_itemssize= , argsarg_itemsconst=true , cond = "expr" , target=target_name:target_name

More than one codelet directive can be added to a function in order to specify different uses or different execution contexts However, there can be only one codelet directive for a given call site label

The callsite directive specifies how the use a codelet at a given point in the program

The syntax of the directive is:

#pragma hmpp <grp_label> codelet_label callsite , asynchronous , argsarg_itemssize= , argsarg_itemsadvancedload=true|false , argsarg_itemsaddr="expr" , argsarg_itemsnoupdate=true

An example is shown here :

/ declaration of the codelet / #pragma hmpp simple1 codelet, argsoutvio=inout, target=CUDA static void matvecint sn, int sm, float invsm, float inmsnsm, float outv outvi = temp; int mainint argc, char argv matvecn, m, myinc, inm, myoutv;

In some cases, a specific management of the data throughout the application is required CPU/GPU data movements optimization, shared variables

The group directive allows the declaration of a group of codelets The parameters defined in this directive are applied to all codelets belonging to the group The syntax of the directive is:

#pragma hmpp <grp_label> group , version = <major><minor><micro> , target = target_name:target_name , cond = “expr”

Data transfers directives to optimize communication overheadedit

When using a HWA, the main bottleneck is often the data transfers between the HWA and the main processor
To limit the communication overhead, data transfers can be overlapped with successive executions of the same codelet by using the asynchronous property of the HWA

  • allocate directive

The allocate directive locks the HWA and allocates the needed amount of memory

#pragma hmpp <grp_label> allocate ,argsarg_itemssize=
  • release directive

The release directive specifies when to release the HWA for a group or a stand-alone codelet

#pragma hmpp <grp_label> release
  • advancedload directive

The advancedload directive prefetches data before the remote execution of the codelet

#pragma hmpp <grp_label> codelet_label advancedload ,argsarg_items ,argsarg_itemssize= ,argsarg_itemsaddr="expr" ,argsarg_itemssection= ,asynchronous
  • delegatedstore directive

The delegatedstore directive is a synchronization barrier to wait for an asynchronous codelet execution to complete and to then download the results

#pragma hmpp <grp_label> codelet_label delegatedstore ,argsarg_items ,argsarg_itemsaddr="expr" ,argsarg_itemssection=
  • Asynchronous Computations

The synchronize directive specifies to wait until the completion of an asynchronous callsite execution For the synchronize directive, the codelet label is always mandatory and the group label is required if the codelet belongs to a group

#pragma hmpp <grp_label> codelet_label synchronize
  • Example

In the following example, the device initialization, memory allocation and upload of the input data are done only once outside the loop and not in each iteration of the loop

The synchronize directive allows to wait for the asynchronous execution of the codelet to complete before launching another iteration Finally the delegatedstore directive outside the loop uploads the sgemm result

int mainint argc, char argv #pragma hmpp sgemm advancedload, argsvin1;vin2;vout, argsm,n,k,alpha,beta for j = 0 ; j < 2 ; j ++ #pragma hmpp sgemm delegatedstore, argsvout #pragma hmpp sgemm release

Sharing data between codeletsedit

Those directives map together all the arguments sharing the given name for all the group

The types and dimensions of all mapped arguments must be identical

The map directive maps several arguments on the device

#pragma hmpp <grp_label> map, argsarg_items

This directive is quite similar as the map directive except that the arguments to be mapped are directly specified by their name The mapbyname directive is equivalent to multiple map directives

#pragma hmpp <grp_label> mapbyname ,variableName+

Global variableedit

The resident directive declares some variables as global within a group Those variables can then be directly accessed from any codelet belonging to the group This directive applies to the declaration statement just following it in the source code

The syntax of this directive is:

#pragma hmpp <grp_label> resident , args::var_nameio=in|out|inout , args::var_namesize= , args::var_nameaddr="expr" , args::var_nameconst=true

The notation ::var_name with the prefix ::, indicates an application’s variable declared as resident

Acceleration of regionsedit

A region is a merge of the codelet/callsite directives The goal is to avoid code restructuration to build the codelet Therefore, all the attributes available for codelet or callsite directives can be used on regions directives

In C language:

#pragma hmpp <MyGroup> label region , argsarg_itemsio=in|out|inout , cond = "expr"< , argsarg_itemsconst=true , target=target_name:target_name , argsarg_itemssize= , argsarg_itemsadvancedload=true|false , argsarg_itemsaddr="expr" , argsarg_itemsnoupdate=true , asynchronous , private=arg_items


The OpenHMPP Open Standard is based on HMPP Version 23 May 2009, CAPS entreprise

The OpenHMPP directive-based programming model is implemented in:

  • CAPS Compilers, CAPS Entreprise compilers for hybrid computing
  • PathScale ENZO Compiler Suite support the NVIDIA GPUs

OpenHMPP is used by HPC actorswho in Oil & Gas,citation needed Energy,citation needed Manufacturing,citation needed Finance,citation needed Education & Researchcitation needed

See alsoedit

  • Parallel computing
  • OpenACC
  • OpenCL


  1. ^ Dolbeau, Romain; Bihan, Stéphane; Bodin, François 4 October 2007 HMPP: A Hybrid Multi-core Parallel Programming Environment PDF Workshop on General Purpose Processing on Graphics Processing Units Retrieved 14 January 2014 

External linksedit

  • CAPS Entreprise SAS and PathScale, Inc to Jointly Collaborate on Making HMPP a New Open Standard
  • How Hardware Will Shape Languages By David Chisnall
  • Code acceleration with HMPP By ICHEC Irish Center for High-End Computing
  • Expérience de programmation avec HMPP By IRMA Institut de Recherche Mathématique Avancée - FORTRAN examples
  • Directive-based Heterogeneous Programming - A GPU-Accelerated RTM Use Case By TOTAL Technical and Scientific Center and CAPS Entreprise
  • HMPP Port By CEA Commissariat à l'Energie Atomique et aux Energies Alternatives for PRACE Partnership for Advanced Computing in Europe

OpenHMPP Information about



OpenHMPP Information Video

OpenHMPP viewing the topic.
OpenHMPP what, OpenHMPP who, OpenHMPP explanation

There are excerpts from wikipedia on this article and video

Random Posts

Social Accounts

Facebook Twitter VK
Copyright © 2014. Search Engine