Post Multipart Form Data in Python with Requests: Flask File Upload Example

Post Multipart Form Data in Python with Requests: Flask File Upload Example

In this tutorial we'll demonstrate how to upload a file from a Python server to another server by sending a POST request with multipart/form-data using the Python requests library.

Originally published at  techiediaries.com on 11 Mar 2019

We'll be using two servers. The server that receives the file doesn't need to be a Python server but since we'he previously created one with Django in this tutorial, let's use it instead of re-inventing the wheel.

Note: Typically we upload files from a client to a server but in this tutorial, we'll see how we can upload files from a server to another web server using Python and the Requests library.

Open a new terminal and create and activate a virtual environment:

$ python3 -m venv .env
$ source .env/bin/activate

Next, clone the GitHub repository and install the dependencies:

$ git clone https://github.com/techiediaries/django-rest-file-upload.git server2
$ cd server2
$ pip install -r requirments.txt

Next, run the server using the following commands:

$ python manage.py makemigrations
$ python manage.py migrate
$ python manage.py runserver

The server will be available from the 127.0.0.1:8000 and will expose an /upload endpoint that accepts a POST request.

Note: This server has also CORS enabled which means it can accept requests from different domains so make sure to enable CORS if you are using any other server.
Creating the Flask Server

Now, let's proceed to create the uploading Python server that will make use of the Requests library to send a POST requests to the 127.0.0.1:8000/upload endpoint for uploading a file between two servers.

Installing requests

Let's install the requests library using pip:

$ pip install requests

Installing Flask

We'll be using Flask; a single file and lightweight web framework for creating the Python server that uploads the file. First install flask using pip:

$ pip install flask

Next, create a server.py file and add the following code:

import os
from flask import Flask, request, render_template
import requests

app = Flask(name)

@app.route('/handle_form', methods=['POST'])
def handle_form():
print("Posted file: {}".format(request.files['file']))
file = request.files['file']
return ""

@app.route("/")
def index():
return render_template("index.html");

if name == "main":
app.run(host='0.0.0.0', port=8080, debug=True)

We create a / route for rendering the index.html template that will display a form and /handle_formroute that will process the multipart form, get the uploaded file from the requests.files[] array and return. We'll use this method to send the form to the django server using the requests library.

Next, create a templates folder and add an index.html file with the following code:

<!DOCTYPE html>
<html>

<head>
<title>Upload New File</title>
</head>

<body>
<h1>Upload Files</h1>

&lt;form action="handle_form" method="post" enctype="multipart/form-data"&gt;
    &lt;input type="file" name="file"&gt;
    &lt;input type="submit" value="Upload"&gt;
&lt;/form&gt;

</body>

</html>

We create a form of multipart/form-data encoding type that has a file field for selecting a file from the hard drive.

Sending Files with the Requests Library

The handle_form() of our flask application receives the posted file but doesn't save it. Instead, we'll use the requests library to upload it to the django server.

Simply change the handle_form() method as follows:

@app.route('/handle_form', methods=['POST'])
def handle_form():
print("Posted file: {}".format(request.files['file']))
file = request.files['file']
files = {'file': file.read()}
r = requests.post("http://127.0.0.1:8000/upload/", files=files)

if r.ok:
    return "File uploaded!"
else:
    return "Error uploading file!"

We get the posted form from the request.Files array, next we use the requests.post() method to upload the file to the other server using a POST request. If the requests is successful, r.ok will be True.

Next, run the server using the following command:

$ python server.py

Your Python server will be available from the 127.0.0.1:8080 address.

If you select a file and upload it, you should have the file uploaded in the media folder of the django server.

Conclusion

In this tutorial, you've seen how you can use Python and the requests library to upload a file from a server to another server.

Originally published at  techiediaries.com on 11 Mar 2019

===================================================================

Thanks for reading :heart: If you liked this post, share it with all of your programming buddies! Follow me on Facebook | Twitter

Learn More

☞ Complete Python Bootcamp: Go from zero to hero in Python 3

☞ Python and Django Full Stack Web Developer Bootcamp

☞ Python for Time Series Data Analysis

☞ Python Programming For Beginners From Scratch

☞ Beginner’s guide on Python: Learn python from scratch! (New)

☞ Python for Beginners: Complete Python Programming

☞ Django 2.1 & Python | The Ultimate Web Development Bootcamp

☞ Python eCommerce | Build a Django eCommerce Web Application

☞ Python Django Dev To Deployment

Python Django Tutorial | Django Course

Python Django Tutorial | Django Course

🔥Intellipaat Django course: https://intellipaat.com/python-django-training/ 👉This Python Django tutorial will help you learn what is django web development &...

This Python Django tutorial will help you learn what is django web development & application, what is django and introduction to django framework, how to install django and start programming, how to create a django project and how to build django app. There is a short django project as well to master this python django framework.

Why should you watch this Django tutorial?

You can learn Django much faster than any other programming language and this Django tutorial helps you do just that. Our Django tutorial has been created with extensive inputs from the industry so that you can learn Django and apply it for real world scenarios.

Developing Restful APIs with Python, Django and Django Rest Framework

Developing Restful APIs with Python, Django and Django Rest Framework

This article is a definitive guide for starters who want to develop projects with RESTful APIs using Python, Django and Django Rest Framework.

This article is a definitive guide for starters who want to develop projects with RESTful APIs using Python, Django and Django Rest Framework.

Introduction
  • Django is a web framework written in Python
  • Python is an interpreted high-level programming language for general-purpose programming
  • API or Application Programming Interface is a set of rules and mechanisms by which one application or component interacts with the others
  • REST or Representational State Transfer is a software architecture

REST APIs

As described in a dissertion by Roy Fielding,

REST is an "architectural style' that basically exploits the existing technology and protocols of the web.
In simple definition, it is the data representation for a client in the format that is suitable for it.

Hence, RESTful + API is a commonly used terminology for the implementation of such architecture and constraints (eg. in web services).

Here is an example GET request from GitHub's API

$ curl https://api.github.com/users/joshuadeguzman

You will see an output similar to this

{
  "login": "joshuadeguzman",
  "id": 20706361,
  "node_id": "MDQ6VXNlcjIwNzA2MzYx",
  "avatar_url": "https://avatars1.githubusercontent.com/u/20706361?v=4",
  "gravatar_id": "",
  "url": "https://api.github.com/users/joshuadeguzman",
  "html_url": "https://github.com/joshuadeguzman",
  "followers_url": "https://api.github.com/users/joshuadeguzman/followers",
  "following_url": "https://api.github.com/users/joshuadeguzman/following{/other_user}",
  "gists_url": "https://api.github.com/users/joshuadeguzman/gists{/gist_id}",
  "starred_url": "https://api.github.com/users/joshuadeguzman/starred{/owner}{/repo}",
  "subscriptions_url": "https://api.github.com/users/joshuadeguzman/subscriptions",
  "organizations_url": "https://api.github.com/users/joshuadeguzman/orgs",
  "repos_url": "https://api.github.com/users/joshuadeguzman/repos",
  "events_url": "https://api.github.com/users/joshuadeguzman/events{/privacy}",
  "received_events_url": "https://api.github.com/users/joshuadeguzman/received_events",
  "type": "User",
  "site_admin": false,
  "name": "Joshua de Guzman",
  "company": "@freelancer",
  "blog": "https://joshuadeguzman.me",
  "location": "Manila, PH",
  "email": null,
  "hireable": true,
  "bio": "Android Engineer at @freelancer. Building tools for humans.",
  "public_repos": 75,
  "public_gists": 2,
  "followers": 38,
  "following": 10,
  "created_at": "2016-07-28T15:19:54Z",
  "updated_at": "2019-06-16T10:26:39Z"
}

Shown above is a data set in JSON format.

JSON or JavaScript Object Notation is an open-standard file format that uses human-readable text to transmit data objects consisting of attribute–value pairs and array data types.
Other formats include XML, INI, CSV, etc. But today, JSON is widely use for its structure is intuitive, making it comfortable to read and map domain objects no matter what programming language is being used.

Python and Django

Python, according to its creator, Guido van Rossum, is a

high-level programming language, and its core design philosophy is all about code readability and a syntax which allows programmers to express concepts in a few lines of code.
Python uses english like words representation (eg. for methods, reserve keywords and control flow) that makes it easier for any beginner to jump right into it. It also features dynamic type system meaning it verifies the type safety of program at runtime. It also does automatic memory management.

print(5 + 5) # This will result to 10

Django is a high-level Python Web Framework that enables developers to deliver projects on time with clean and pragmatic design.

Its flagship features include a design for fast development, a secure and scalable product.

Quick Django Overview

Django's way of propagating changes to your database schema is by means of its migration modules.

Sample User model

from django.db import models

class User(models.Model):
    first_name = models.CharField(max_length=50)
    middle_name = models.CharField(max_length=50)
    last_name = models.CharField(max_length=50)

    def __str__(self):
        return self.name

If any changes are made on your models, run makemigrations

$ python manage.py makemigrations

Finally, you can synchronize the database with the set of models and migrations

$ python manage.py migrate

REST APIs with Django Rest Framework

DRF or Django REST Framework is a powerful and flexible toolkit for building Web APIs. It helps the developers to not reinvent the wheel by rolling out complex and solid REST API from scratch by themselves. Because when your projects become more and more complex, you will soon realise the need of using DRF or other helpful rest framework.

1. Installation & Project Setup

Create project directory

$ mkdir djangoapi

Install virtualenv via pip

A virtual environment enables a project to have additional libraries or changes in packages within its environment without disturbing global or libraries of other environments.

pip is a package management system used to install and manage software packages written in Python.

$ pip install virtualenv

To create an environment folder in your project's directory

$ cd djangoapi
$ virtualenv venv

To activate the environment

$ source venv/bin/activate

To undo these changes to your path, simply run deactivate. More on virtualenv.

Install django, djangorestframework

$ pip install django
$ pip install djangorestframework

Creating a django project

$ django-admin startproject blog

Running your project

$ python manage.py runserver

System check identified no issues (0 silenced).

You have 15 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.

August 16, 2018 - 09:58:36
Django version 2.1, using settings 'blog.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

The unapplied migrations refer to the default migration files included when you start a django project.

To synchronize these migration files, simply run migrate

$ python manage.py migrate

Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying sessions.0001_initial... OK

The default database in our project is currently set to SQLite named db.sqlite3.

Creating a django project's app

$ cd blog
$ python manage.py startapp posts

The project structure should look like

$ find .
./posts
./posts/migrations
./posts/migrations/__init__.py
./posts/models.py
./posts/__init__.py
./posts/apps.py
./posts/admin.py
./posts/tests.py
./posts/views.py
./db.sqlite3
./blog
./blog/__init__.py
./blog/__pycache__
./blog/__pycache__/settings.cpython-36.pyc
./blog/__pycache__/wsgi.cpython-36.pyc
./blog/__pycache__/__init__.cpython-36.pyc
./blog/__pycache__/urls.cpython-36.pyc
./blog/settings.py
./blog/urls.py
./blog/wsgi.py
./manage.py

2. Model

Each model instance is a definitive source of the information about your data. In general, each model pertains to a single table in your database.

# djangoapi/blog/posts/models.py
from django.db import models

# Create your models here.

class Post(models.Model):
    title = models.CharField(max_length=255)
    content = models.TextField()
    is_featured = models.BooleanField(default=False)

    def __str__(self):
        return self.name

__str__ is called by the str() built-in function and by the print statement to compute the "informal" string representation of an object.
If you try running makemigrations, django won't see those changes yet.

$ No changes detected

To solve this, add your posts app to your project's installed apps.

# djangoapi/blog/blog/settings.py

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'posts' # Add it here
]

To continue with the migration of models

$ python manage.py makemigrations

Migrations for 'posts':
  posts/migrations/0001_initial.py
    - Create model Post

$ python manage.py migrate

Operations to perform:
  Apply all migrations: admin, auth, contenttypes, posts, sessions
Running migrations:
  Applying posts.0001_initial... OK


3. Serialization

Serializers allow data structure or object state to be translated into a format that can be stored or transmitted and be reconstructed later on.

Create API's serializers.py and views.py files and isolate them like this

# posts/api
posts/api/serializers.py
posts/api/views.py

# posts/migrations
posts/migrations/

# posts
posts/admin.py
posts/apps.py
posts/models.py
posts/tests.py
posts/views.py
# posts/api/serializers.py

from ..models import Post
from rest_framework import serializers

class PostSerializer(serializers.ModelSerializer):
    class Meta:
        model = Post
        fields = ('title', 'content', 'is_featured') # if not declared, all fields of the model will be shown

In this tutorial we have used ModelSerializer, more on this.

4. Views

A view function, or view for short, is a Python function that takes a Web request and returns a Web response.

# posts/api/views.py

from ..models import Post
from . import serializers
from rest_framework import generics, status
from rest_framework.response import Response

class PostListView(generics.ListAPIView):
    queryset = Post.objects.all()
    serializer_class = serializers.PostSerializer

As seen above, ListAPIView is used for read-only endpoints to represent a collection of model instances.

In this code snippet, we use generics view methods from the rest_framework, more on this.

5. URLs

This is where we setup our routes or URL paths to our designated views in which we expect specific responses for each.

# posts/urls.py

from django.urls import path
from . import views
from .api import views

urlpatterns = [
    path('', views.PostListView.as_view(), name=None)
]

6. Finalizing Setup

Ensure that the rest_framework is added to our project's apps.

# djangoapi/blog/blog/settings.py

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'rest_framework', # Add it here
    'posts'
]

7. Django Admin

Since we haven't setup our POST requests yet, we will be populating the database through django's admin panel.

To do that, create a superuser account admin with password 1234password.

$ python manage.py createsuperuser --email [email protected] --username admin

Password:
Password (again):
This password is too common.
Bypass password validation and create user anyway? [y/N]: y
Superuser created successfully.

Register the model in the admin panel.

# posts/admin.py

from django.contrib import admin
from .models import Post

# Register your models here.
admin.site.register(Post)

That's it. Visit the admin panel and update posts model's records. More on this.

8. Testing our API

$ python manage.py runserver
GET /api/v1/posts/
HTTP 200 OK
Allow: GET, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

[
    {
        "title": "Example Post #1",
        "content": "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.",
        "is_featured": false
    },
    {
        "title": "Example Post #2",
        "content": "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.",
        "is_featured": true
    }
]

Great. Now it's time for us to update our views and finish the standard CRUD operations.

9. Adding more views

POST is a method used for creating (sometimes updating) a resource in the database.

# posts/api/views.py

from ..models import Post
from . import serializers
from rest_framework import generics, status
from rest_framework.response import Response

class PostCreateView(generics.CreateAPIView):
    queryset = Post.objects.all()
    serializer_class = serializers.PostSerializer

    def create(self, request, *args, **kwargs):
        super(PostCreateView, self).create(request, args, kwargs)
        response = {"status_code": status.HTTP_200_OK,
                    "message": "Successfully created",
                    "result": request.data}
        return Response(response)

Most often, we separate List and Create view classes when we want to expose a list of data set while easily preventing a certain request to POST or create a resource in the database for that specific List view.

Usecase always varies for apps, you are opt to use ListCreateAPIView or even ViewSets for combining the logic for a set of related views.

Optional: Since we want to display the data in a more systematic way, we override create method and map our inline custom response handler.

Adding more views with methods GET, PATCH, DELETE to handle a specific blog post detail.

class PostDetailView(generics.RetrieveUpdateDestroyAPIView):
    queryset = Post.objects.all()
    serializer_class = serializers.PostSerializer

    def retrieve(self, request, *args, **kwargs):
        super(PostDetailView, self).retrieve(request, args, kwargs)
        instance = self.get_object()
        serializer = self.get_serializer(instance)
        data = serializer.data
        response = {"status_code": status.HTTP_200_OK,
                    "message": "Successfully retrieved",
                    "result": data}
        return Response(response)

    def patch(self, request, *args, **kwargs):
        super(PostDetailView, self).patch(request, args, kwargs)
        instance = self.get_object()
        serializer = self.get_serializer(instance)
        data = serializer.data
        response = {"status_code": status.HTTP_200_OK,
                    "message": "Successfully updated",
                    "result": data}
        return Response(response)

    def delete(self, request, *args, **kwargs):
        super(PostDetailView, self).delete(request, args, kwargs)
        response = {"status_code": status.HTTP_200_OK,
                    "message": "Successfully deleted"}
        return Response(response)

10. Updating URLs

# posts/urls.py

from django.urls import path
from . import views
from .api import views

urlpatterns = [
    path('', views.PostListView.as_view(), name=None),
    path('create/', views.PostCreateView.as_view(), name=None),
    path('<int:pk>/', views.PostDetailView.as_view(), name=None)
]

Now you can send requests to your API via Postman, your app or do a GETrequests from your browser, examples:

POST /api/v1/posts/create/
HTTP 200 OK
Allow: POST, OPTIONS
Content-Type: application/json
Vary: Accept

{
    "status_code": 200,
    "message": "Successfully created",
    "result": {
        "csrfmiddlewaretoken": "rnSUN3XOIghnXA0yKghnQgxg0do39xhorYene5ALw3gWGThK5MjG6YjL8VUb7v2h",
        "title": "Creating a resource",
        "content": "Howdy mate!"
    }
}
GET /api/v1/posts/1/
HTTP 200 OK
Allow: GET, PUT, PATCH, DELETE, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

{
    "status_code": 200,
    "message": "Successfully retrieved",
    "result": {
        "title": "Sample Post",
        "content": "Sample Post Content",
        "is_featured": false
    }
}

That's it. You have successfully managed to develop RESTful APIs with DRF! Cheers!

Source code

Available on GitHub.