S3 to Frame.io Copy Using Lambda

S3 to Frame.io Copy Using Lambda

The workflow is built using the Serverless Framework. Therefore, even people without much knowledge of the services involved should be able to get it running with a few commands. Goals

I wanted a solid solution for copying an entire S3 bucket into a Frame.io project that would keep the folder structure and support any file size. Frame.io provides a great Zapier integration for the same task, but it's limited in what file size it can handle and does not recreate folders.

The workflow is built using the Serverless Framework. Therefore, even people without much knowledge of the services involved should be able to get it running with a few commands.

Goals

  • Quick copy from S3 to Frame.io
  • Handle any file and bucket size
  • Preserve folder structure

Architecture

  • API Gateway triggers the workflow from a POST request
  • Main Lambda verifies project info
  • Copy Lambda copies S3 -> Frame.io
  • Copy restarts itself if Lambda timeouts

The copy is done by generating a presigned S3 URL to your asset and sending it Frame.io for ingest. Doing it this way greatly reduces Lambda processing time since it only does simple HTTP requests and no handling of the actual bytes in the file.

Setting it up

  • Make sure you have npm and docker installed and running
  • Clone the github repo and install the Serverless framework
git clone https://github.com/strombergdev/s3-to-frameio-lambda-copy.git
cd s3-to-frameio-lambda-copy 
npm install -g serverless
sls plugin install -n serverless-python-requirements

frameio lambda s3 serverless-framework post-production

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Deploy AWS Lambda (Node.js) with Serverless and read data from S3

Create S3 Bucket and upload csv file; Install Serverless Framework locally on your machine; Create a Serverless project (Node.js/Express.js) ...

Developing Lambda Applications using the AWS Serverless Application Model (SAM) Framework

Developing, testing, and deploying Lambda applications using the AWS Serverless Application Model (SAM) framework. Developing Lambda Applications using the AWS Serverless Application Model (SAM) Framework

Configuring Lambda Event Destinations with the Serverless Framework

Learn how to configure AWS Lambda Event Destinations in two additional lines of code inside of your serverless.yml file.

DIY — Build a Serverless Framework With 152 Lines of Code.

Have you ever asked yourself how a framework has been built to do a magic job for you? In this article, we will be together in 3 minutes to learn HOWTO to build a framework. This framework will deploy your function code into AWS Lambda or GCloud Functions service as you choose.

WebSockets on Demand With AWS Lambda, Serverless Framework, and Go

Lambda functions and WebSockets — how the opposites meet in the cloud. In this article, we describe how to implement a WebSocket server using AWS API Gateway and Lambda functions.