File Connectors for Data Sources

This system provides a standardized approach to managing file storage operations across different backends. Each data connector includes a default file connector, and custom implementations can be added as needed by extending the FileConnector class present in Core.

Key methods of FileConnector:

• uploadFileContent(fileTag, dataType, binary): Uploads binary data to storage.
• downloadFileContent(fileTag, dataType): Retrieves binary data using a file tag.
• deleteFileContent(fileTag, dataType): Deletes files from storage.

Configuration properties, such as file paths or credentials, can be accessed via the getConfig(key) method. Custom file connectors need to be registered in the FileConnectorsRegistry. For example:

FileConnectorsRegistry.register(
  'customFileConnector',
  (configGetter) => new CustomFileConnector(configGetter)
);

---

Local Disk File Connector

The local disk file connector stores files directly on the server’s file system.

• To use this, add a filePath property to the data connector configuration:

{
    "data": {
        "host": "<host>",
        "user": "<user>",
        "password": "<password>",
        "database": "<database>",
        "schema": "<schema>",
        "filePath": "<file path>"
    }
}

---

Azure Blob Storage

To store files in Azure Blob Storage instead of the default file storage location, you need to update the oConfig file with specific properties.

Steps to Enable Azure Blob Storage

1. Add the file_connector Property

   • To change the file storage from the default location to Azure Blob Storage, add the file_connector property and set it to "azureBlobStorage" in the oConfig file.

2. Add the azure Object

   • Along with the file_connector property, add the azure object. This object contains the authentication type (authType) and the necessary properties required for Azure Blob Storage.

Configuration Example

Here is an example configuration for the oConfig file:

{
    "file_connector": "azureBlobStorage",
    "azure": {
        "authType": "<authType> SAS, SASToken or SPN",
        "blobUrl": "<blob-url>",
        "container": "<container-name>",
        "accountName": "<account-name>",
        "accountKey": "<account-key>",
        "tenantId": "<tenant-id>",
        "clientId": "<client-id>",
        "clientSecret": "<client-secret>",
        "sasToken": "<sas-token>",
        "pathFromRoot": "<optional-path-from-root>"
    }
}

Parameter	Auth Type	Description
authType	All	The authentication type. Options: "SAS", "SPN", or "SASToken".
blobUrl	SAS, SASToken	The URL of the Azure Blob Storage endpoint (e.g., emulator or Azure URL).
container	All	The name of the blob container where files will be stored.
accountName	SAS	The name of the Azure Storage account.
accountKey	SAS	The access key for the Azure Storage account.
sasToken	SASToken	The SAS token for accessing the blob container.
tenantId	SPN	The Azure Active Directory Tenant ID of the service principal.
clientId	SPN	The Client ID of the service principal.
clientSecret	SPN	The Client Secret of the service principal.
pathFromRoot	All	Optional. Specifies a subfolder within the container for file storage.