Schedule Adaptor Data Exports and Routines

Please make sure you have read through the Get Started/Overview before continuing down this document.

Also make sure you understand the concepts of:

• Connector Adaptors.
• Connector Routines.

When the Connector Windows Service first starts it loads a list of the enabled data exports and routines for each adaptor, which are each timed to run at specified date times. The list of adaptor data exports and routines is configured by users through the Connector Application.

Running Scheduled Data Exports

When the time hits for a scheduled data export to run, the connector host will call on the relevant adaptor to retrieve the required data from the adaptor's configured accounting/ERP system, and push it to the adaptor's configured Ecommerce system. After the export is finished the results of the data export will be made available in the schedule history visible witin the Connector application, and optionally an email notification can be sent out with the results.

Running Scheduled Routines

For adaptor routines, when the time hits for a scheduled routine to run, the connector host will call on the relevant adaptor, which will run its routine and perform the required processing. After the routine is finished the results of the routine will be made available in the schedule history within the Connector application. To learn what each adaptor's routines do please refer to the specific adaptor's documention (through the links in the left hand navigation on this page).

Run Once Off Data Exports/Routines

Both data exports and routines can be manually run once off from the Connector application. "Run once off" means that a user has explicitly clicked the button that calls for the data export or routine to run at the current time, which may be needed for testing, analysis, or updating data outside of normal scheduled periods.

Intervals and Rescheduling

Once a scheduled data export or routine has been run, the connector host will then reschedule it to run again based on the interval period set up. Intervals can be set up on daily, hourly or on a minute basis eg. run a "product" data export every 5 days, starting from 20/01/2014 at 2am.

We recommend that you set scheduling so that data exports/routine date times do not clash, and allow enough time for the processing of one to complete before the next one starts.
This recommendation reduces too many resources being used on your machine and external systems at one given time, which could cause errors or serious malfunctions of software to occur.

Missed Scheduling

Sometimes it is possible that a scheduled data export or routine is missed, either because the connector host was not running at the time, or because too many processes were being run at the same time on the machine. If this occurs then when the connector next starts up it can reset the next scheduled date of the data export/routine to a time in the future (based on the next interval period from the current date time).

Disable From Running

Scheduled data exports and routines can be disabled from running, this allows them to stay in the schedule list, and be reactivated at a later time. You may wish to do this if the system the export or routine uses is currently down, under maintenance, or if you don't wish data to be modified in external systems.
Conversely the scheduled data exports/routines can be completely removed from the adaptor when there is no longer a need to be scheduled.

When exporting data using an adaptor it is important that the different data exports are run in a particular order. This is required because certain data depends on other data already being stored in the external systems.

Below is a diagram that shows the data dependencies that each export has to one another. In the diagram if a data export contains a line with an arrow pointing to another export, then it is dependent on the pointed to export to run first. For example the Product Pricing export is dependent on the Products export and Price Levels exports running before the Product Pricing export is run.

If the data exports are not run in the correct order then the external systems that are receiving the data may not be able to correctly import and/or store all the given data and its relationships.

When setting up data export schedule times for an adaptor, ensure that you factor in these dependencies.

Notes:

• The Customer Accounts data export can run before the Warehouses and Product Stock data export if Customer Accounts are not required to be assigned to specific warehouses.
• The Customer Accounts data export can run before the Sales Representatives data export if no sales representatives/employees are assigned to specific customer accounts.
• The Categories data export can run before the Products data export if there is not a need to assign products to categories.
• The Warehouses and Product Stock data export can run before the Products data export if it is not required for products to be assigned to warehouses.
• Product Account Pricing data export can run before the Customer Account Product Contracts if it is there are no contracts associated with customer accounts.

To understand more about the data that is exported out in each Data Export, please read the Data Export Types document, linked to below.

Below are the list of data exports that each Ecommerce system the Connector supports for sending the exported data from an adaptor to. See Data Export Types for more details on what each data export does.

SQUIZZ.com Platform

• Price Levels
• Sell Units
• Surcharges
• Taxcodes
• Products
• Product Pricing
• Product Quantity Pricing
• Product Account Pricing
• Product Images
• Product Attributes
• Product Alternate Codes
• Product Stock
• Locations and Location Stock
• Categories
• Sales Representatives
• Customer Accounts
• Customer Account Addresses
• Purchasers
• Supplier Accounts
• Supplier Account Addresses
• Sales Orders

TOTECS Platform

• Price Levels
• Sell Units
• Surcharges
• Taxcodes
• Products
• Product Pricing
• Product Quantity Pricing
• Product Account Pricing
• Product Attachments
• Product Images
• Product Attributes
• Product Combinations
• Product Flags
• Product Alternate Codes
• Product Stock
• Product Kits
• Item Groups
• Item Relations
• Locations and Location Stock
• Categories
• Sales Representatives
• Customer Accounts
• Customer Account Addresses

Connector Data Set

• Sell Units
• Products
• Product Attachments
• Product Images
• Product Attributes
• Product Combinations
• Product Flags
• Product Alternate Codes
• Product Stock
• Product Kits
• Item Groups
• Item Relations
• Locations and Location Stock
• Categories

When a data export runs for an adaptor, there is the ability to set it to incrementally or fully export all the data that was obtained from the Accounting/ERP System. Incremental means that only the differences to the data since the last export are pushed up to the Ecommerce system. Full means that all the records in the export are pushed up to the Ecommerce system regardless of whether the system already has the same data or not. By sending only incremental changes to the Ecommerce system it can dramatically reduce the amount of data transferred between the two systems, reducing bandwidth, processing, and associated costs with moving large sets of data around.

In the Connector each adaptor has a cache database that allows it to store the records returned from the Accounting/ERP system that are being exported out. For a data export when the Data Transfer Mode is set to Incremental, the adaptor will fill its cache, and compare what's in the cache with the next full set of records are being exported out. Only the record differences will then be pushed to the Ecommerce system. After the data has been exported the cache will be updated with the new records ready for the next time the data export runs. If no changes are found, then the data is not exported to the Ecommerce system, and the connector will report as such in its message logs.

We highly recommend that you set data exports to use the incremental data transfer mode. The connector is smart enough to know that if the something goes wrong during any stage of caching, or pushing data up to the Ecommerce system, it will mark the cache as being out of integrity for the given data export, and force the full amount of data to be exported whilst cleaning out the cache for next time.

There may be times that you may wish to run a full data export, such as when you think the Ecommerce system's data has become out of integrity.

When a data export is run for the first time in incremental mode, it will always export the full amount of data, and then cache the data. If the data export is run as a full export, it will not cache its data before or after pushing the data to the Ecommerce system.

Please note that as of 29th August 2014, only the Product Account Pricing data export supports caching and intemental data exports. If any other data export has its data transfer mode set to incremental, this will be ignored and the full amount of data will be exported.

To immediately run a data export for an adaptor follow these steps:

1. Open the Connector application.
2. In the Adaptors and Messages tab, in the Adaptors panel, in the Schedule column click the Exports/Routines button against the relevant adaptor.
3. In the Adaptor Export/Routine Schedule window, in the Data Exports Tab select one of the data exports in the list that you want to run.
4. Click the Run Once Off button.

The Connector application will communicate to the Connector Host Service and ask it to call the adaptor's data export to run. If the data export successfully obtains its data then it will try to export the data to the selected Ecommerce platform. Once finished it will create a History Log advising if the data export successfully completed or not, and the details of the attempt. Additionally the data export may also raise log messages within the Connector Messages table on the Connector application's main window, under the Adaptors and Messages tab. View these logs to find out more information about what the data export did. It's recommended to set the Show Log Types filter to ALL before running a data export if you wish to see all log messages, including debugging for a data export.

If you wish to repeatedly run a data export repeatedly at regular times then read the "Adding a Data Export to the Adaptor's Schedule" section on this page.

Once an adaptor routine has been created for an adaptor then follow these steps below to call it to run. Before running a routine make sure you understand what the routine does by reading its documentation. If you are unsure on how to create an adaptor routine then read the targeting adaptor's documentation on how to do so.

1. Open the Connector application.
2. In the Adaptors and Messages tab, in the Adaptors panel, in the Schedule column click the Exports/Routines button against the relevant adaptor.
3. In the Adaptor Export/Routine Schedule window, in the Routines Tab select one of the routines in the list that you want to run.
4. Click the Run Once Off button.
5. Click on the History Log tab to see if it had run successfully.
6. Click on the Refresh Log History button to update the history log table. If the routine doesn't not appear there then it may still be running. If this is the case then wait for longer until it has finished, then press the Refresh Log History button again.
7. View the history log reord to check if the routine ran successfully.

Some routines may send email notifications when they run. Routines may also output logs in the Connector Messages table. Read the documentation of the specific routine being run to determine what information it will output.

To schedule a data export to be scheduled for an adaptor in the connector, follow the following steps:

1. Open the Connector application.
2. In the Adaptors and Messages tab, in the Adaptors panel, in the Schedule column click the Exports/Routines button against the relevant adaptor.
3. In the Adaptor Export/Routine Schedule window, in the Data Exports Tab select one of the data exports in the list.
4. Click the Add to Schedule button.
5. In the Configure Scheduled Data Export/Routine panel configure the following settings:
   • Enabled: tick to enable the data export.
   • Frequency: enter a number then choose from DAY, HOUR, MINUTE, this sets how the data export gets rescheduled once it runs.
   • Next Run Date Time: Set the date and time for when the data export will next run.
   • Send Email Notification Report: Choose Yes to have an email notification sent out once the data export runs. Set it to Only When The Export/Routine Fails to only have an email notification sent out of the data export fails to complete successfully.
   • Data Transfer Mode: Set to Incremental (Reccommeded) when only changes since the last export are to be pushed up to the server. Set to Full to export the entire set of data to the server. 
6. Click the Add button. A new row with the settings will appear in the Scheduled Data Exports/Routines tab.
7. Click the Reload Scheduler button. Pressing the button will tell the Connector Host to reload all the scheduled data exports and routines, including the new data export.

To schedule a routine for an adaptor in the connector, follow the following steps:

1. Open the Accounting Connector application.
2. In the Adaptors and Messages tab, in the Adaptors panel, in the Schedule column click the Exports/Routines button against the relevant adaptor.
3. In the Adaptor Export/Routine Schedule window, click on the Routines tab.
4. Select the desired routine to schedule. If no routines are in the list then you need to create a routine within the adaptor's settings windows
5. Click the Add to Schedule button.
6. In the Configure Scheduled Data Export/Routine panel configure the following settings:
   • Enabled: tick to enable the scheduled routine.
   • Frequency: enter a number then choose from DAY, HOUR, MINUTE, this sets how the routine gets rescheduled once it runs.
   • Next Run Date Time: Set the date and time for when the routine will next run.
   • Send Email Notification Report: Choose Yes to have an email notification sent out once the routine runs. Set it to Only When The Export/Routine Fails to only have an email notification sent out of the routine fails to complete successfully.
7. Click the Add button. A new row with the settings will appear in the Scheduled Data Exports/Routines tab.
8. Click the Reload Scheduler button. Pressing the button will tell the Connector Host to reload all the scheduled data exports and routines, including the new routine.

To schedule a routine for an adaptor in the connector, follow the following steps:

1. Open the Accounting Connector application.
2. In the Adaptors and Messages tab, in the Adaptors panel, in the Schedule column click the Exports/Routines button against the relevant adaptor.
3. In the Adaptor Export/Routine Schedule window, in the Scheduled Data Exports/Routines list press the Modify button against the relevant scheduled data export/routine.
4. In the Configure Scheduled Data Export/Routine panel configure the following settings:
   • Enabled: tick to enable the scheduled routine.
   • Frequency: enter a number then choose from DAY, HOUR, MINUTE, this sets how the routine gets rescheduled once it runs.
   • Next Run Date Time: Set the date and time for when the routine will next run.
   • Send Email Notification Report: Choose Yes to have an email notification sent out once the routine runs. Set it to Only When The Export/Routine Fails to only have an email notification sent out of the routine fails to complete successfully.
   • Data Transfer Mode: Set to Incremental (Reccommended) when only changes since the last export are to be pushed up to the server. Set to Full to export the entire set of data to the server. 
5. Click the Update button. The existing row will be updated within the Scheduled Data Exports/Routines tab.
6. Click the Reload Scheduler button. Pressing the button will tell the Connector Host to reload all the scheduled data exports and routines, including the updated routine.

To schedule a routine for an adaptor in the connector, follow the following steps:

1. Open the Connector application.
2. In the Adaptors and Messages tab, in the Adaptors panel, in the Schedule column click the Exports/Routines button against the relevant accounting adaptor.
3. In the Adaptor Export/Routine Schedule window, in the Scheduled Data Exports/Routines list press the Modify button against the relevant scheduled data export/routine.
4. In the Configure Scheduled Data Export/Routine panel untick the Enabled option.
5. Click the Update button. The existing row will be updated within the Scheduled Data Exports/Routines tab.
6. Click the Reload Scheduler button. Pressing the button will tell the Connector Host to reload all the scheduled data exports and routines, excluding the updated routine from being scheduled.

To schedule a routine for an adaptor in the connector, follow the following steps:

1. Open the Connector application.
2. In the Adaptors and Messages tab, in the Adaptors panel, in the Schedule column click the Exports/Routines button against the relevant adaptor.
3. In the Adaptor Export/Routine Schedule window, in the Scheduled Data Exports/Routines list press the Remove button against the relevant scheduled data export/routine. The row will be removed for the scheduled data export/routine in the Scheduled Data Exports/Routines tab.
4. Click the Reload Scheduler button. Pressing the button will tell the Connector Host to reload all the scheduled data exports and routines, excluding the deleted data export/routine.

To schedule a routine for an adaptor in the connector, follow the following steps:

1. Open the Accounting Connector application.
2. In the Adaptors and Messages tab, in the Accounting System Adaptors panel, in the Schedule column click the Exports/Routines button against the relevant accounting adaptor.
3. In the Accounting Adaptor Export/Routine Schedule window, click on the History Log tab. You will then see a list of data exports/routines previously run with the following information:
   • Data Export/Routine: Name of the data export or routine.
   • Status: Either SUCCESS or FAILURE.
   • Start Date: Date and time that the data export or routine started running.
   • Finish Date: Date and time that the data export or routine finished running.
   • Details: Any information related to the running of the data export. This may contain error information if the data export/routine fails to suceed.
4. Click the Refresh Log History button to reload the History Log table. If a data export/routine runs after the logs first load in the table you will need to click this button to view its results.
5. Click the Clear History Log button to clear all logs of the history. We recommend doing this when the logs get over 500 records in size (otherwise the logs may start taking longer to load).
6. Click the View button in a given row to view all the details of the history log record in a window.