Hertha  Walsh

Hertha Walsh

1604210160

OpenCV + CUDA + AWS EC2 + (No More Tears)

By default, there is no need to enable OpenCV with CUDA for GPU processing, but during production, when you have heavy OpenCV manipulations to do on image/video files, we can make use of the OpenCV CUDA library to make those operations to run on GPU rather than CPU and it saves a lot of time.

It was not easy as it is said to connect the OpenCV library to enable it with CUDA, I had to go through a painful process for a week to establish the connection properly, also its both time & money consuming process. So this time I want to record the overall process for my future, as well as for others.

For the demonstration, I am renting an EC2 instance with a p3.8xlarge instance in the AWS, which has 4 Nvidia GPUs.

Image for post

Source — AWS EC2 Pricing

So if you need any help in starting an EC2 instance for the first time, you can refer to my previous post on Step by Step Creation of an EC2 Instance in AWS and Access it via Putty & WinSCP and during the process select the GPU instance you require.

Now after ssh-ing into the instance, before we get into the process we need to install a lot of packages to make the environment ready.

_Note: I have consolidated all the commands I ran from start to end and added them at the bottom. If you are more curious find them here in this __link _and follow along.

Run the below commands one after another on your instance and also I have attested the screenshots to compare the outputs against mine.

All the screenshots used hereafter are sourced by the author.

Table of Contents:

  1. Install OpenCV Dependencies, Nvidia CUDA driver, CUDA toolkit
  2. Download OpenCV Source Code
  3. Configure Python Virtual Environment
  4. Determine Your CUDA Architecture Version
  5. Configure OpenCV with Nvidia GPU Support
  6. Compile OpenCV and Create a Symbolic link
  7. References
  8. History of Commands

Step 1: Install OpenCV Dependencies, Nvidia CUDA driver, CUDA toolkit.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install build-essential cmake unzip pkg-config
sudo apt-get install gcc-6 g++-6
sudo apt-get install screen
sudo apt-get install libxmu-dev libxi-dev libglu1-mesa libglu1-mesa-dev
sudo apt-get install libjpeg-dev libpng-dev libtiff-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev
sudo apt-get install libopenblas-dev libatlas-base-dev liblapack-dev gfortran
sudo apt-get install libhdf5-serial-dev
sudo apt-get install python3-dev python3-tk python-imaging-tk
sudo apt-get install libgtk-3-dev
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update
sudo apt-get install nvidia-driver-418
sudo reboot

#opencv-in-ubuntu #opencv-python #cuda #nvidia #opencv #ubuntu

What is GEEK

Buddha Community

OpenCV + CUDA + AWS EC2 + (No More Tears)
Hertha  Walsh

Hertha Walsh

1604210160

OpenCV + CUDA + AWS EC2 + (No More Tears)

By default, there is no need to enable OpenCV with CUDA for GPU processing, but during production, when you have heavy OpenCV manipulations to do on image/video files, we can make use of the OpenCV CUDA library to make those operations to run on GPU rather than CPU and it saves a lot of time.

It was not easy as it is said to connect the OpenCV library to enable it with CUDA, I had to go through a painful process for a week to establish the connection properly, also its both time & money consuming process. So this time I want to record the overall process for my future, as well as for others.

For the demonstration, I am renting an EC2 instance with a p3.8xlarge instance in the AWS, which has 4 Nvidia GPUs.

Image for post

Source — AWS EC2 Pricing

So if you need any help in starting an EC2 instance for the first time, you can refer to my previous post on Step by Step Creation of an EC2 Instance in AWS and Access it via Putty & WinSCP and during the process select the GPU instance you require.

Now after ssh-ing into the instance, before we get into the process we need to install a lot of packages to make the environment ready.

_Note: I have consolidated all the commands I ran from start to end and added them at the bottom. If you are more curious find them here in this __link _and follow along.

Run the below commands one after another on your instance and also I have attested the screenshots to compare the outputs against mine.

All the screenshots used hereafter are sourced by the author.

Table of Contents:

  1. Install OpenCV Dependencies, Nvidia CUDA driver, CUDA toolkit
  2. Download OpenCV Source Code
  3. Configure Python Virtual Environment
  4. Determine Your CUDA Architecture Version
  5. Configure OpenCV with Nvidia GPU Support
  6. Compile OpenCV and Create a Symbolic link
  7. References
  8. History of Commands

Step 1: Install OpenCV Dependencies, Nvidia CUDA driver, CUDA toolkit.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install build-essential cmake unzip pkg-config
sudo apt-get install gcc-6 g++-6
sudo apt-get install screen
sudo apt-get install libxmu-dev libxi-dev libglu1-mesa libglu1-mesa-dev
sudo apt-get install libjpeg-dev libpng-dev libtiff-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev
sudo apt-get install libopenblas-dev libatlas-base-dev liblapack-dev gfortran
sudo apt-get install libhdf5-serial-dev
sudo apt-get install python3-dev python3-tk python-imaging-tk
sudo apt-get install libgtk-3-dev
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update
sudo apt-get install nvidia-driver-418
sudo reboot

#opencv-in-ubuntu #opencv-python #cuda #nvidia #opencv #ubuntu

Lindsey  Koepp

Lindsey Koepp

1603458624

A Beginner's Guide to Amazon EC2 (Elastic Compute Cloud)

Amazon EC2 is a service that allows you businesses to run their application in AWS cloud. You can easily set up a virtual machine and perform all your computation on it. In this post, we will teach you the basics of Amazon EC2.

Introduction

EC2 is one of the oldest and fundamental services of AWS. Though everything today is serverless, you still need to know EC2. EC2 allows you to do the following things :

  • Rent virtual machines (EC2 itself)
  • Store data on virtual drives (EBS)
  • Distribute load across the machines (ELB)
  • Scale the services using an auto-scaling group (ASG)

Amazon Machine Image – AMI

AMIs are the templates from which virtual servers (EC2 instance) get created.

In simple terms, AMIs are the collection of the operating system, architecture (32-bit or 64-bit) with the softwares that will be running on that operating system.

For example:

  • Amazon Linux AMI is the one that comes with a Linux operating system and many packages of the Linux that one might need.

  • Windows AMI comes with Windows as its operating system and software one might need.

By choosing AMI you basically decide the software part of your EC2 machine. You can use a single AMI and launch multiple instances from it.

#aws #aws-cloud #aws-iam #aws-certification #aws-top-story #ec2 #virtual-machine #amazon-web-services

Seamus  Quitzon

Seamus Quitzon

1601341562

AWS Cost Allocation Tags and Cost Reduction

Bob had just arrived in the office for his first day of work as the newly hired chief technical officer when he was called into a conference room by the president, Martha, who immediately introduced him to the head of accounting, Amanda. They exchanged pleasantries, and then Martha got right down to business:

“Bob, we have several teams here developing software applications on Amazon and our bill is very high. We think it’s unnecessarily high, and we’d like you to look into it and bring it under control.”

Martha placed a screenshot of the Amazon Web Services (AWS) billing report on the table and pointed to it.

“This is a problem for us: We don’t know what we’re spending this money on, and we need to see more detail.”

Amanda chimed in, “Bob, look, we have financial dimensions that we use for reporting purposes, and I can provide you with some guidance regarding some information we’d really like to see such that the reports that are ultimately produced mirror these dimensions — if you can do this, it would really help us internally.”

“Bob, we can’t stress how important this is right now. These projects are becoming very expensive for our business,” Martha reiterated.

“How many projects do we have?” Bob inquired.

“We have four projects in total: two in the aviation division and two in the energy division. If it matters, the aviation division has 75 developers and the energy division has 25 developers,” the CEO responded.

Bob understood the problem and responded, “I’ll see what I can do and have some ideas. I might not be able to give you retrospective insight, but going forward, we should be able to get a better idea of what’s going on and start to bring the cost down.”

The meeting ended with Bob heading to find his desk. Cost allocation tags should help us, he thought to himself as he looked for someone who might know where his office is.

#aws #aws cloud #node js #cost optimization #aws cli #well architected framework #aws cost report #cost control #aws cost #aws tags

Hire AWS Developer

Looking to Hire Professional AWS Developers?

The technology inventions have demanded all businesses to use and manage cloud-based computing services and Amazon is dominating the cloud computing services provider in the world.

Hire AWS Developer from HourlyDeveloper.io & Get the best amazon web services development. Take your business to excellence with our best AWS developer that will serve you the benefit of different cloud computing tools.

Consult with experts: https://bit.ly/2CWJgHyAWS Development services

#hire aws developer #aws developers #aws development company #aws development services #aws development #aws

Christa  Stehr

Christa Stehr

1598408880

How To Unite AWS KMS with Serverless Application Model (SAM)

The Basics

AWS KMS is a Key Management Service that let you create Cryptographic keys that you can use to encrypt and decrypt data and also other keys. You can read more about it here.

Important points about Keys

Please note that the customer master keys(CMK) generated can only be used to encrypt small amount of data like passwords, RSA key. You can use AWS KMS CMKs to generate, encrypt, and decrypt data keys. However, AWS KMS does not store, manage, or track your data keys, or perform cryptographic operations with data keys.

You must use and manage data keys outside of AWS KMS. KMS API uses AWS KMS CMK in the encryption operations and they cannot accept more than 4 KB (4096 bytes) of data. To encrypt application data, use the server-side encryption features of an AWS service, or a client-side encryption library, such as the AWS Encryption SDK or the Amazon S3 encryption client.

Scenario

We want to create signup and login forms for a website.

Passwords should be encrypted and stored in DynamoDB database.

What do we need?

  1. KMS key to encrypt and decrypt data
  2. DynamoDB table to store password.
  3. Lambda functions & APIs to process Login and Sign up forms.
  4. Sign up/ Login forms in HTML.

Lets Implement it as Serverless Application Model (SAM)!

Lets first create the Key that we will use to encrypt and decrypt password.

KmsKey:
    Type: AWS::KMS::Key
    Properties: 
      Description: CMK for encrypting and decrypting
      KeyPolicy:
        Version: '2012-10-17'
        Id: key-default-1
        Statement:
        - Sid: Enable IAM User Permissions
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:root
          Action: kms:*
          Resource: '*'
        - Sid: Allow administration of the key
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:user/${KeyAdmin}
          Action:
          - kms:Create*
          - kms:Describe*
          - kms:Enable*
          - kms:List*
          - kms:Put*
          - kms:Update*
          - kms:Revoke*
          - kms:Disable*
          - kms:Get*
          - kms:Delete*
          - kms:ScheduleKeyDeletion
          - kms:CancelKeyDeletion
          Resource: '*'
        - Sid: Allow use of the key
          Effect: Allow
          Principal:
            AWS: !Sub arn:aws:iam::${AWS::AccountId}:user/${KeyUser}
          Action:
          - kms:DescribeKey
          - kms:Encrypt
          - kms:Decrypt
          - kms:ReEncrypt*
          - kms:GenerateDataKey
          - kms:GenerateDataKeyWithoutPlaintext
          Resource: '*'

The important thing in above snippet is the KeyPolicy. KMS requires a Key Administrator and Key User. As a best practice your Key Administrator and Key User should be 2 separate user in your Organisation. We are allowing all permissions to the root users.

So if your key Administrator leaves the organisation, the root user will be able to delete this key. As you can see **KeyAdmin **can manage the key but not use it and KeyUser can only use the key. ${KeyAdmin} and **${KeyUser} **are parameters in the SAM template.

You would be asked to provide values for these parameters during SAM Deploy.

#aws #serverless #aws-sam #aws-key-management-service #aws-certification #aws-api-gateway #tutorial-for-beginners #aws-blogs