A RESTful API to ingest data from external source


Steps to ingest data using API as an endpoint


Following are the steps to ingest data via this API :


1. Request the API with the following URL , to receive a presigned URL

URL : https://api.<environment-name>-neo.bcone.com/s3presignedurl/v1/ingestdata


Prerequisites to request the API

  • Pass the X-API key in the header section
  • Pass the encrypted key in the body section

Sample Request Format


Method : POST

Header :

x-api-key:<key obtained from neosupport@bcone.com>

Content-Type:application/json

Body :

{

"key": "<encrypted key> ",

"fileName": "<input filename>"

Note 1 : An Key : Encrypted key will be shared with the user

[A combination of AmazonS3 Raw bucket, Site, Feed, Filetype, Frequency type in the format of Amazon

S3 Raw bucket/Site/Feed/Filetype/Frequency of the user’s data pipeline will be encrypted and shared with the user ]

Note 2 : filename :Name of the file to be ingested

Note 3 : Encrypted key and filename are mandatory

Success Response

{

"preSignedUrl": <preSignedUrl returned by the API>,

"status": "SUCCESS"

}


Failed Response

{

"errorMessage": <an error message is returned based on the type of error>,

"status": "FAILED"

}


2. Use the generated presigned URL to ingest the data into S3

URL : <preSignedUrl returned by the API as shown in the first step>

Method : PUT

Request : Request the API with a file object (.csv ,.pdf )

Sample Request Code in Angular

const req = new HttpRequest('PUT', url, file, { reportProgress: true });

this.http.request(req);

Note :"URL in the above code is presigned Url
          "File : any file object

Success Response : Status Code 2xx: Success

Failed Response : Status Code 4xx : Client Error