Celery for Task Management with Flask and SQS

Assumed background knowledge

This article assumes the reader has familiarity with Python, Flask, Celery, and AWS SQS.

Introduction

The fundamental thing to grasp when building a Flask app that utilizes Celery for asynchronous task management is that there are really three parts to consider, outside of the queue and result backends. These are (1) the Flask instance, which is your web or micro-service frontend, (2) the Celery instance, which feeds tasks to the queue, and (3) the Celery worker, which pulls tasks off the queue and completes the work. The Flask and Celery instances are deployed together and work in tandem at the interface of the application. The Celery worker is deployed separately and works effectively independent from the instances.

At first glance setting up an application for using these three components appears very simple. However, the complication arises when attempting to implement the Flask instance and Celery instance using the Flask application factory pattern, because the approach causes a circular import issue.

The objective of this article of to:

1. clarify how to initialize these three parts of the Flask+Celery service
2. explain how to containerize the pieces of these services for deployment to a production environment
3. point out a some notes on best practices

#flask #task-management #python #celery #sqs

What is GEEK

Buddha Community

Celery for Task Management with Flask and SQS

Assumed background knowledge

This article assumes the reader has familiarity with Python, Flask, Celery, and AWS SQS.

Introduction

The fundamental thing to grasp when building a Flask app that utilizes Celery for asynchronous task management is that there are really three parts to consider, outside of the queue and result backends. These are (1) the Flask instance, which is your web or micro-service frontend, (2) the Celery instance, which feeds tasks to the queue, and (3) the Celery worker, which pulls tasks off the queue and completes the work. The Flask and Celery instances are deployed together and work in tandem at the interface of the application. The Celery worker is deployed separately and works effectively independent from the instances.

At first glance setting up an application for using these three components appears very simple. However, the complication arises when attempting to implement the Flask instance and Celery instance using the Flask application factory pattern, because the approach causes a circular import issue.

The objective of this article of to:

1. clarify how to initialize these three parts of the Flask+Celery service
2. explain how to containerize the pieces of these services for deployment to a production environment
3. point out a some notes on best practices

#flask #task-management #python #celery #sqs

Jupyter Notebook Kernel for Running ansible Tasks and Playbooks

Ansible Jupyter Kernel

The Ansible Jupyter Kernel adds a kernel backend for Jupyter to interface directly with Ansible and construct plays and tasks and execute them on the fly.

Demo

Installation:

ansible-kernel is available to be installed from pypi but you can also install it locally. The setup package itself will register the kernel with Jupyter automatically.

From pypi

pip install ansible-kernel
python -m ansible_kernel.install

From a local checkout

pip install -e .
python -m ansible_kernel.install

For Anaconda/Miniconda

pip install ansible-kernel
python -m ansible_kernel.install --sys-prefix

Usage

Local install

    jupyter notebook
    # In the notebook interface, select Ansible from the 'New' menu

Container

docker run -p 8888:8888 benthomasson/ansible-jupyter-kernel

Then copy the URL from the output into your browser:
http://localhost:8888/?token=ABCD1234

Using the Cells

Normally Ansible brings together various components in different files and locations to launch a playbook and performs automation tasks. For this jupyter interface you need to provide this information in cells by denoting what the cell contains and then finally writing your tasks that will make use of them. There are Examples available to help you, in this section we'll go over the currently supported cell types.

In order to denote what the cell contains you should prefix it with a pound/hash symbol (#) and the type as listed here as the first line as shown in the examples below.

#inventory

The inventory that your tasks will use

#inventory
[all]
ahost ansible_connection=local
anotherhost examplevar=val

#play

This represents the opening block of a typical Ansible play

#play
name: Hello World
hosts: all
gather_facts: false

#task

This is the default cell type if no type is given for the first line

#task
debug:

#task
shell: cat /tmp/afile
register: output

#host_vars

This takes an argument that represents the hostname. Variables defined in this file will be available in the tasks for that host.

#host_vars Host1
hostname: host1

#group_vars

This takes an argument that represents the group name. Variables defined in this file will be available in the tasks for hosts in that group.

#group_vars BranchOfficeX
gateway: 192.168.1.254

#vars

This takes an argument that represents the filename for use in later cells

#vars example_vars
message: hello vars

#play
name: hello world
hosts: localhost
gather_facts: false
vars_files:
    - example_vars

#template

This takes an argument in order to create a templated file that can be used in later cells

#template hello.j2
{{ message }}

#task
template:
    src: hello.j2
    dest: /tmp/hello

#ansible.cfg

Provides overrides typically found in ansible.cfg

#ansible.cfg
[defaults]
host_key_checking=False

Examples

You can find various example notebooks in the repository

Using the development environment

It's possible to use whatever python development process you feel comfortable with. The repository itself includes mechanisms for using pipenv

pipenv install
...
pipenv shell

---

Author: ansible
Source Code:  https://github.com/ansible/ansible-jupyter-kernel
License: Apache-2.0 License

#jupyter #python 

Originscale Order Management System

Originscale order management software helps to manage all your orders across channels in a single place. Originscale collects orders across multiple channels in real-time - online, offline, D2C, B2B, and more. View all your orders in one single window and process them with a simple click.

#order management system #ordering management system #order management software #free order management software #purchase order management software #best order management software

Digital Assets Management Software Solution | AI-based Assets Management System

A Digital Asset Management System makes it easier to store, manage, and share all of your digital assets on cloud-based storage.

We help you to build Digital Asset Management (DAM) systems with your precise business requirements, whether you want one for maintaining management, production management, brand management systems, or implementing with your sales department with the digital assets it needs.

To learn more about how the Digital Asset Management system will help your business, email us at hello@techavidus.com

#digital assets management #assets management solution #digital asset management system #production management #brand management

Revenue Cycle Management Software Services and Custom Integration - SISGAIN

Revenues come day in day out and it becomes strenuous to keep a track of them. With the help of Revenue cycle management software, one is able to perform the hospital revenue cycle management in Oklahoma, USA in a much simplified and easy manner. Our skilful developers and engineers created the healthcare revenue cycle management software that is convenient to use by its users and meets the customers requirement. We happen to be one of the notable revenue cycle management companies, facilitating the needs of our customers and being efficient and useful in performance. For more information call us at +18444455767 or email us at hello@sisgain.com

#revenue cycle management #revenue cycle management software #revenue cycle management companies #hospital revenue cycle management #revenue cycle management services #revenue cycle management solutions