Instance ManagementData Structure

Data Structure Definition

Configure data structure for entities.

Overview

The Data Structure Definition screen allows you to:

  • Create new Entities
  • View the list of Entities uploaded to Fynapse
  • Delete single or multiple Entities

For more details on the Data Structure, refer to Data Structure.

General Information

Uploading Data Structure Definition

This article provides information on how to create a Defined Entity on the Data Structure Definition screen.

For a detailed structure of the JSON snippet the can be used to upload Entities in the Configuration Data JSON file, refer to the Data Structure Definition section in Configuration Data chapter.

Data Structure Definition Constraints

The maximum numbers of user defined Entities stored in Fynapse in Finance Data Service is 100.

Defined Entities are divided into two types, Transaction and Reference, based on the type of data stored in an Entity:

  • Reference - determines that the Entity has a reference data type
  • Transaction - determines that the Entity has a Transaction data type; these are accounting data that cannot be changed

Attributes

Defined Entities are comprised of attributes which fall into two categories, primitive and complex:

Primitive attributes:

  • UUID
    Please note that the UUID inputs are not generated automatically. You need to generate them either via a third party generator or in your upstream system and include them in the ingest file with the rest of the input data.
  • BOOLEAN
  • TIMESTAMP - the format used in Fynapse is: yyyy-mm-dd HH:mm:ss. This format is hardcoded and cannot be changed.
  • DATE - the date format Fynapse is: yyyy-mm-dd. This format is hardcoded and cannot be changed.
  • TEXT
  • INT
  • DECIMAL - the decimal attribute operates with the following parameters: scale 18 digits and precision 3 digits. This format is hardcoded and cannot be changed.
  • HIGH-PRECISION DECIMAL

Complex attributes:

  • Inline Entity - this is an entity attribute type, i.e. a child entity nested within another entity.
  • List - this is an attribute type which can store an ordered collection of data, primitive or complex, e.g. Inline Entities. The data contained within the List will be ordered in accordance with the order they were supplied to Fynapse. An example of this is a ‘person’ Entity with list of ‘children’, ordered from youngest to oldest.
    A special example of this are Protected entities Journal and Journal Line, with Journal Entity containing a list of Journal Line Entities. Their order is determined by Fynapse.

Please note:

  • InLine type complex attribute can only be defined as a nested attribute of a List type attribute; you cannot define InLine type attribute as a standalone attribute
  • It is not possible to define a List type attribute as nested attribute of another List type attribute

Effective From

The Effective From date is a technical field that is mapped do a dateField defined in a Reference Entity.

If you don’t define a dateField in your Reference Entity or the dateField attribute is left empty in your ingestion file, a default Effective From date will be assigned by the system. This means that the Primary Key will be the only identifier of reference data and the newest version of the data, as identified off of the Primary Key, will be used for enrichment.

If you decide to populate the dateField, then when there are multiple versions of reference data the data with Effective From date effective at the time of the Posting Date of an ingested transaction event will be used for enrichment for this transaction.

For Defined Entities, Effective From date is mapped to the Systemic Field Date Type:

  • Core Date for Journal Line Definition
  • Effective Date for Business Event Definition
    For Business Event Definition this is optional. If you don’t configure the Effective Date field, the Effective From date will by default use the current System Date during enrichment.

For Transaction type Reference Data Entities, the dateField works just like Core Date for Journal Line Definition and Effective Date for Business Event Definition, i.e. it is mapped to the Effective From date technical field and used during enrichment to add correct Reference Data to the ingested Transaction Event. If no dateField is configured for a Transaction type Reference Data Entity, the Reference Data are added based off the current system date.

Data Retention Policy

Data retention policy allows you to periodically purge outdated records to prevent excessive storage consumption. By default all stored Entities have this parameter set to Indefinite.

We recommend that you set a custom retention period for Reference Entities populated by the extraction process and used as lookups during Flow processing, since such data may be regenerated each time the process runs, so periodically purging outdated records will prevent excessive storage consumption. When a retention policy is defined, the records will be automatically deleted after the specified number of days from their creation date.

For stored Transaction type Entities, the retention policy is always set for the Indefinite period. You cannot set a fixed period for this type of Entities.

This parameter can be set to a custom value:

  • when creating an Entity on the Data Structure Definition screen
  • in the Configuration Data JSON file. If you don’t set this parameter, the Entity will by default have an Indefinite retention period.

Validations

You can define validations in references for Entities that will be used in the enrichment process.

Uploading Data Structure Definition

This article provides information on how to create a validations for Lookup references when creating a new Entity on the Data Structure Definition screen.

For full details of all types of validations and creating validations for both Protected and Defined Entities, refer to Validations.

For a detailed structure of the JSON snippet that can be used to upload validations in the Configuration Data JSON file, refer to the Data Structure Definition section in Configuration Data chapter.

How Validation Suggestions Are Generated on the Setup Validations Screen?

When you are creating a new Entity, the system will analyze the fields you create and suggest validations against fields in already existing Entities in the FDS, based on similarities in Field Label and Data Type.

So if any of the created fields has the Field Label and Data Type that corresponds to the Field Label and Data Type of a field in an Entity that already exists in the FDS, the system will suggest creating a lookup reference between these two fields. This mechanism applies to both scenarios for creating a new Entity, create manually and create from sample data.

The suggestions are created based off of the following guidelines:

Uploading Data Structure Definition

The analysis is performed for Entities in the same namespace.

Only Entities created on the Data Structure Definition screen, i.e. Reference and Transaction Entities with flat structures that store data in the FDS.

Which fields are analyzed:

  • Fields with the following primitive attribute types:
    • text
    • UUID
    • integer

Which fields are skipped:

  • Fields declared as Primary Keys
  • The field declared as effectiveDate
  • Fields with the following primitive attribute types:
    • TIMESTAMP DATE
    • BOOLEAN
    • DECIMAL
    • HIGH_PRECISION_DECIMAL
  • Fields whose name matches a known audit field: userid, username, createdby, updatedby, modifiedby, createdat, updatedat, creationtime, updatetime, lastupdated, lastmodified, lastmodifiedby, insertedby, inserteddate.

How fields are matched:

Both the field name and the field label are analyzed independently and the closer match is suggested on the screen.

Two fields are treated as equivalent when:

  • They’re names are identical.
  • One is a recognised abbreviation of the other (e.g. attr → attribute), provided the abbreviated name or label is at least 3 characters long and the full name or label is at least 1.5× longer.
  • They are the same length (≥ 4 characters) and differ by exactly one character substitution (e.g. trage → trade) or one adjacent transposition — two neighbouring characters swapped (e.g. tarde → trade).

The single closest match per input field is suggested.

Data Structure Definition Screen

The Data Structure Definition screen comprises of a grid with the following information:

  • Entity name - unique name of the Entity
  • Entity status - status of the Entity
    • Creating - a temporary status displayed in the grid when creating is in progress
    • Created - the Entity has been created
    • Error creating - an error occurred during creating the Entity
    • Deleting - a temporary status displayed when deleting is in progress
    • Error deleting - an error occurred during deleting the Entity
  • Type - type of the Entity, can be either Reference or Transaction
  • Stored? - information whether the Entity is an Entity that stores data in the FDS or is a transient Entity used by extracts; by default set to Yes; for stored Entities, you will also see the information of the length of the retention period
  • Added by - username of the user who uploaded the Entity
  • Added on - timestamp of when the Entity was uploaded

The Actions column has the following actions:

  • View entity structure - redirects you to the details screen of the given Entity
  • Delete entity structure - deletes the selected Entity

Additionally, you can select one or multiple Entities to delete using the checkboxes in the first column of the left. Once selected, the Delete entity button becomes active and you can delete the selected Entities.

Entity Details Screen

The Entity details screen comprises of two sections.

Entity Information

The Entity information ssection comprises of the following information:

  • Entity name - unique name of the Entity
  • Entity type - type of the Entity, can be either Reference or Transaction
  • Store data - information whether the Entity is an Entity that stores data in the FDS or is a transient Entity used by extracts
  • Retention policy - for stored Entities, information if a retention policy was kept as default Indefinite or if the length of the retention period if set to custom

Field information

The Field information ssection comprises of a grid with the following information:

  • Id - ordinal number of item in the grid
  • Primary key - information whether the field is a Primary Key field
  • Field label - configured label of the field
  • Field type - data type of the field
  • Mandatory? - information whether the field is mandatory; additionally if a field is a Primary Key field or an Effective Date field, there will be information, PK or ED, in brackets

Tutorials

  1. Go to Data Structure Definition.
  2. Click the Create new entity button.
  3. In the dialog, select Create from scratch. You will be redirected to the Create new entity screen.
  4. In the Entity information section:
    1. Type a unique Entity name.
    2. Select the Entity type. You can create a:
      1. Transaction Entity - an Entity which stores transaction data
      2. Reference Entity - an Entity which stores reference data
    3. The Store data switch determines if your Entity is a Transient Entity. You can configure your Entity to be a regular Entity which stores data in the FDS, switch set to Yes, or a Transient Entity, i.e. an Entity which is a transitional Entity for data generated by an extract which is not stored in the FDS, switch set to No.
    4. The Retention policy allows you to choose if your stored Entity will keep the Indefinite retention period, which is the default for all stored Entities and the only option available for stored Transient Entities; or for stored Reference Entities you can set a Custom retention period.
  5. In the Field information section, click the + Add new field button to add the fields that will comprise your Entity. Add the following information about the field:
    Please note these settings can be later changed in the grid.

    1. Field label - functional name of the field that will appear in the system

    2. Field type - data type of the field

    3. Options - you can choose the field to be Mandatory or comprise the Primary Key of your Entity.

      For date type fields you also have an option to choose it as an Effective date field, required for efficient versioning of data in Reference Entities. Only one date field can be set as the Effective Date field, if you try to choose another one a message asking if you want to change the existing Effective Date field will appear.

      Check the relevant box for your new field.

      If you have a list in your Entity, you will also have the option to add the new field to the list using the Add to [list_name] option.

      Please note, that if you choose a field to be a Primary Key field or an Effective Date field, it will be checked as mandatory by default.

  6. You can create a list with nested primitive fields in your Entity structure:
    1. Click the Add new list button.
    2. In the dialog, type the List name. Remember this name has to be unique within the Entity.
    3. You have two options for adding fields to a list:
      1. Select available existing fields to add to the list.
      2. Create an empty list and add new fields. In this scenario, you follow the instruction from point 5 above and check the Add to [list_name] option.
    4. Click the Create list button.
  7. You can also upload a .csv file with sample data that the system will use to create fields in your Entity. These can be new fields added to the ones your already created or they can replace the fields you added manually.
    Please note that when you upload a .CSV file, if a column has data it will be by default set as Mandatory. 
  8. You can move the order of the fields by dragging the hamburger in the left-hand column.
  9. Once you are satisfied with the structure click the Create entity button.
  10. You will be redirected to the Setup Validations screen. On the screen you will see a list of suggested Lookup references. These suggestions are generated for new fields from your new Entity, which have the same Field Label and Data Type as a field in an Entity that already exists in the FDS. Please note that if the system doesn’t detect any matching fields, this screen will be empty. Skip points a and b.
    1. Go over the list of suggested references, use the check icon to accept a reference or the x icon to discard.
    2. You can choose to accept all suggested references by using the Accept all button, or discard all by using the Discard all button.
    3. Click the Save and create new entity button once you’ve gone over the suggested references list.
    4. If you don’t want to create any references and don’t reject the suggested references, then when you click the Save and create new entity button an additional dialog will appear, asking if you want to Save without reviewing. Click the Save without reviewing button to confirm you don’t want to analyze the suggested references.
  11. In the dialog, click the Create entity button to create the Entity or Cancel to discard your changes.
  1. Go to Data Structure Definition.
  2. Click the Create new entity button.
  3. In the dialog, select Upload sample data.
  4. Use drag-and-drop or Select file to upload your .csv file.
  5. Click the Create new entity button. You will be redirected to the Create new entity screen.
  6. On the Create new entity screen, fill out the information in the Entity information section
    1. Type a unique Entity name. This field will be pre-filled with the name of the uploaded file.
    2. Select the Entity type. You can create a:
      1. Transaction Entity - an Entity which stores transaction data
      2. Reference Entity - an Entity which stores reference data
    3. The Store data switch determines if your Entity is a Transient Entity. You can configure your Entity to be a regular Entity which stores data in the FDS, switch set to Yes, or a Transient Entity, i.e. an Entity which is a transitional Entity for data generated by an extract which is not stored in the FDS, switch set to No.
    4. The Retention policy allows you to choose if your stored Entity will keep the Indefinite retention period, which is the default for all stored Entities and the only option available for stored Transient Entities; or for stored Reference Entities you can set a Custom retention period.
  7. In the Field information section, check the fields created from the sample data in the grid.
    Please note that when you upload a .CSV file, if a column has data it will be by default set as Mandatory. 
    You can edit the created fields in the grid and add new ones, if required, with the following information about the field:
    Please note these settings can be later changed in the grid.

    1. Field label - functional name of the field that will appear in the system

    2. Field type - data type of the field

    3. Options - you can choose the field to be Mandatory or comprise the Primary Key of your Entity.
      For date type fields you also have an option to choose it as an Effective date field, required for efficient versioning of data in Reference Entities. Only one date field can be set as the Effective Date field, if you try to choose another one a message asking if you want to change the existing Effective Date field will appear.
      Check the relevant box for your new field.

      Note: If you choose a field to be a Primary Key field or an Effective Date field, it will be checked as mandatory by default.

  8. You can move the order of the fields by dragging the hamburger in the left-hand column.
  9. Once you are satisfied with the structure click the Create entity button.
  10. You will be redirected to the Setup Validations screen. On the screen you will see a list of suggested Lookup references. These suggestions are generated for new fields from your new Entity, which have the same Field Label and Data Type as a field in an Entity that already exists in the FDS. Please note that if the system doesn’t detect any matching fields, this screen will be empty. Skip points a and b.
    1. Go over the list of suggested references, use the check icon to accept a reference or the x icon to discard.
    2. You can choose to accept all suggested references by using the Accept all button, or discard all by using the Discard all button.
    3. Click the Save and create new entity button once you’ve gone over the suggested references list.
    4. If you don’t want to create any references and don’t reject the suggested references, then when you click the Save and create new entity button an additional dialog will appear, asking if you want to Save without reviewing. Click the Save without reviewing button to confirm you don’t want to analyze the suggested references.
  11. In the dialog, click the Create entity button to create the Entity or Cancel to discard your changes.

Currently, you cannot edit an Entity directly from UI.

  1. Go to System Configurations > Configuration Data.
  2. Upload a new Configuration Data JSON file containing the modified Entity using the Upload button.
  3. Go to System Configurations > Data Structure > Data Structure Definition. Click the Refresh button to see the new Entity in the grid.
  1. Go to System Configurations > Data Structure > Data Structure Definition.
  2. In the grid, click the row containing an Entity you want to delete.

    Note that deleting an Entity cannot be reverted.

  3. Click the Delete button.
  4. Confirm your action. The selected Entity will be deleted using Schema Repository from the database and as a result from the grid as well.
  5. Click the Refresh button to see the results of deletion.
  1. Go to System Configurations > Data Structure > Data Structure Definition.

Note that deleting an Entity cannot be reverted.

  1. Click the Delete all button.
  2. Confirm your action. All Entities will be deleted using Schema Repository from the database and as a result from the grid as well.
  3. Click the Refresh button to see the results of deletion.