Extraction Logs for Microsoft Dynamics 365

Overview

The Extraction Logs screen allows you to view details of data extracts run in Fynapse to verify, for example, which data was extracted, what is the progress status and who triggered it.

Extraction Logs Screen

The Extraction Logs screen comprises the following elements:

The Refresh button - click it to see if any new extractions were triggered or if the statuses of already triggered ones have changed
The grid - use it to see information about finished and still running data extractions. The grid has the following columns:
- Extract name - an automatically created name of a particular extract file. The name has the following syntax: a configuration name and a start date and time (UTC time). For example: daily_extract_2022-08-05_12-00-00.
- Started on - a date and time of the extraction process start (in a local time zone)
- Uploaded on - a date and time (in a local time zone) when the extract file was sent to a cloud storage or an external system. If the extraction or upload process failed, then there is no information about the date and time.
- Extraction status:
  - In progress - when the extraction process is happening
  - Completed - when the extraction process ended successfully
  - Failed - when the extraction process did not succeed
  - Completed (no data) - when the extraction process has ended successfully but an extract file does not contain any data
- Target status - the sending process goes through three systems and each of them has their statuses:
  - Kafka:
    - Processing - Kafka received data from Fynapse and is processing it
    - Failed - Kafka did not send data to the exporter
    - Duplicated data - the file with extracted data contains only duplicates so it will not be processed further and sent
    - Unknown - the processing failed due to an unknown error
  - Exporter:
    - Exporting - the exporter received data, prepared it for sending, and saved all files containing data
    - Sending - the exporter is sending data
    - Not sent - the exporter was not able to send data to Microsoft Dynamics 365 or the exporter is not receiving any status from Microsoft Dynamics 365
  - Microsoft Dynamics 365:
    - In progress - the exporter can send data to Microsoft Dynamics 365 and it received statuses from Microsoft Dynamics 365, and any of these statuses contain errors
    - Failed - at least one of the statuses received from Microsoft Dynamics 365 contains errors
    - Completed - sending ended successfully (all statuses received from Microsoft Dynamics 365 are without errors)
- Target space - a name of a cloud storage or an external system to which the extract file will be sent
- Executed - the way in which the extraction process was started. It can be done manually (triggered by a user by clicking the Run now button) or automatically (initiated by a system according to a particular schedule).
- Generated by - displays who triggered the extraction process. If the extraction was started by:
  - The system, then a column cell is empty
  - The user, then the username is displayed
- Configuration name - a name of an extract configuration created while defining this configuration. You can click the configuration name to be redirected to these configuration details.
- Details - click the icon to find out more about a particular extraction log
- Actions - click the More button to see all actions that you can perform for the particular extract:
  - Rerun - an action available for extracts that have the Completed or Failed status for the Extraction status. However, the rerun action might not be available for some extracts that have the Failed status due to configuration errors or the fact that the Entity for which the extract was run does not exist anymore. After clicking the Rerun option a selected extract is run again.

Additionally, you can:

Sort columns ascending or descending and set their visibility by clicking the three dot icon next to the names of columns
Filter content within each column to find particular information

In addition, at the bottom of the screen, you can:

See the number of pages that display extraction logs and buttons to navigate between pages
See the information which page you are currently viewing and how many pages are available in total
Decide how many extraction logs you want to display on one page
See how many extraction logs are presented on the page

Extracts Detailed Screen

The extract detailed view screen comprises the following elements:

The Refresh button - click it to see if any new extractions were triggered
Exported chunk name - the name of the particular batch chunk after the aggregation and division to smaller bits
Journal batch number - the identifier of a Journal batch
Message ID - the message ID available for a particular batch in Microsoft Dynamics 365
Sent on - a date and time when Fynapse received the first message from Microsoft Dynamics 365 and was able to send data
Updated on - depending on the stage of processing a date and time displayed in this column can have a different meaning:
- When data is still processing then this is the date and time of the last message received from Microsoft Dynamics 365
- When processing ended then this is the date and time when the processing completed
- When something went wrong while processing then this is the date and time when Microsoft Dynamics 365 stopped sending messages to Fynapse
Processing status - statuses of particular batches from particular stages of the processing sent by Microsoft Dynamics 365:
- Exporting - exporting the file is in progress
- Sending - sending the file is in progress
- Not sent - the file was not sent because Microsoft Dynamics 365 is inaccessible (it can be caused by for example wrong configuration between Fynapse and Microsoft Dynamics 365)
- Enqueued - a file with data was successfully enqueued to the Blob storage
- Dequeued - the file with data was successfully dequeued from the Blob storage
- Acked - the exported file was acknowledged to be downloaded by the external application
- Preprocessing - the export/import operation is pre-processing the request
- PreProcessingError - the export/import operation failed in the pre-processing stage
- Processing - the export/import operation is in progress
- Processed - the export/import operation completed successfully
- ProcessedWithError - the export/import operation was completed with errors
- PostProcessingFailed - the export/import operation failed during post-processing
Action - click the icon to download the particular batch of extracted data from Fynapse in a CSV format

Tutorials

How to View Details of a Particular Extraction Log?

Go to Integration > Extraction Logs.
In the grid find an extract file with details you want to view.
In the Details column, click the chevron. You will be redirected to a new screen displaying details.

To check if the data that you can see is up to date, click the Refresh icon above the grid.

How to Download an Extract Batch?

Go to Integration > Extraction Logs.
In the grid find an extract file from which you want to download data to your computer.
In the Details column, click the chevron.
In the grid find an extract batch that you want to download.
In the Action column, click the icon to download this extract batch.
Select a location where you want to save the file and save it. The extract batch will be downloaded.

How to Rerun an Extract?

Only extracts that have the Completed or Failed status for the Extraction status can be rerun. However, some extracts cannot be rerun due to configuration errors or the fact that the Entity for which the extract was run does not exist anymore. In such a case the rerun action will not be available even if the extract has the Failed status.

Go to Integration > Extraction Logs.
In the grid find an extract file that you want to rerun.
In the Actions column click the More button.
Click Rerun. The extraction process will start and once it is finished you will see a proper extraction log in a new row with a re-run suffix and the extract file will be placed on the defined cloud storage. To see the extraction log in the grid click the Refresh button.

Learn More

Extract Configuration