In this method a custom jar is used to trigger the Restful API . This method helps the user to automate the process of data ingestion.

Prerequisites

  • Encrypted key [A combination of AmazonS3 Raw bucket, Site , Feed , Filetype , Frequency type  in the format of Amazon S3 Raw bucket/Site/Feed/Filetype/Frequency of the user’s data pipeline will be encrypted]will be shared to the user
  • A .ZIP folder that contains Custom Jar , Log configuration file

Steps to ingest data using Custom Jar Method


1. Unzip the API folder and execute the Custom jar using the command

       For Windows : “java -jar upload-file-presigned-0.2.0-SNAPSHOT-jar-with-dependencies.jar D:/json <encrypted key>”

       For Linux  : “java -jar upload-file-presigned-0.2.0-SNAPSHOT-jar-with-dependencies.jar <directory path> <encrypted key>”


2. Post execution of the command , A File watcher is triggered

       [File Watcher : An API based event that monitors file creation on the specified directory ]



3. Upload/Place the input data file in the directory ,monitored by File Watcher



4. A log is created to capture the following event


        User can configure the following attributes using the log file


    1. Log file path
    2. User encrypted key
    3. An event notification upon the successful upload of the file
    4. File Path in which the Raw Data is uploaded

Note : User can also set the size of the log file