Myah  Conn

Myah Conn

1592386200

Tornado and SQLAlchemy

Asynchronous Programming Intro
Asynchronous programming can be a bit tricky with all its jargon but let’s break it down in simple terms. Think of it like performing a lot of tasks around the house. You can either do them one at a time and wait for for each to finish before starting the next (synchronous) or you can start one, then start the other while the first one is going. Imagine you start the laundry. It wouldn’t make sense for you to wait for it to finish before starting to cook dinner. It is much more efficient to get the laundry started in the washing machine, then cook dinner while the laundry is working.

#developer #python #tornado #programming #software-development

What is GEEK

Buddha Community

Tornado and SQLAlchemy
Ruth  Nabimanya

Ruth Nabimanya

1623175920

Introspecting Databases with Airflow and SQLAlchemy

A Tutorial For Iterating Over Automatically Introspected Database Objects

For a recent data migration project that utilized Airflow, I needed to connect to a database and automatically introspect its schemas and tables. One database that needed to be introspected had hundreds of schemas. And each schema had dozens of varying tables, most of which had different columns and column constraints.

In other words, we had database metadata made of of tens- or hundreds of thousands of objects that no one would manually type out into config files or hard-coded Python lists. Knowing that Airflow uses the SQLAlchemy module to communicate with databases as dialect- and engine-agnostically as possible, I had a hunch I could leverage Airflow’s SQLAlchemy connection to the database to generate the lists of schemas, tables, columns, or whatever database objects I needed to access.

My hunch was right! SQLAlchemy and Airflow’s DB API hook make it easy to get a hold of the SQLAlchemy engine for any connected database.

#sqlalchemy #database #introspect #airflow #introspecting databases with airflow and sqlalchemy #sqlalchemy

Myah  Conn

Myah Conn

1592386200

Tornado and SQLAlchemy

Asynchronous Programming Intro
Asynchronous programming can be a bit tricky with all its jargon but let’s break it down in simple terms. Think of it like performing a lot of tasks around the house. You can either do them one at a time and wait for for each to finish before starting the next (synchronous) or you can start one, then start the other while the first one is going. Imagine you start the laundry. It wouldn’t make sense for you to wait for it to finish before starting to cook dinner. It is much more efficient to get the laundry started in the washing machine, then cook dinner while the laundry is working.

#developer #python #tornado #programming #software-development

Ruth  Gleason

Ruth Gleason

1667661600

Marshmallow Sqlalchemy: SQLAlchemy integration with Marshmallow

marshmallow-sqlalchemy

SQLAlchemy integration with the marshmallow (de)serialization library.

Declare your models

import sqlalchemy as sa
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import scoped_session, sessionmaker, relationship, backref

engine = sa.create_engine("sqlite:///:memory:")
session = scoped_session(sessionmaker(bind=engine))
Base = declarative_base()


class Author(Base):
    __tablename__ = "authors"
    id = sa.Column(sa.Integer, primary_key=True)
    name = sa.Column(sa.String, nullable=False)

    def __repr__(self):
        return "<Author(name={self.name!r})>".format(self=self)


class Book(Base):
    __tablename__ = "books"
    id = sa.Column(sa.Integer, primary_key=True)
    title = sa.Column(sa.String)
    author_id = sa.Column(sa.Integer, sa.ForeignKey("authors.id"))
    author = relationship("Author", backref=backref("books"))


Base.metadata.create_all(engine)

Generate marshmallow schemas

from marshmallow_sqlalchemy import SQLAlchemySchema, auto_field


class AuthorSchema(SQLAlchemySchema):
    class Meta:
        model = Author
        load_instance = True  # Optional: deserialize to model instances

    id = auto_field()
    name = auto_field()
    books = auto_field()


class BookSchema(SQLAlchemySchema):
    class Meta:
        model = Book
        load_instance = True

    id = auto_field()
    title = auto_field()
    author_id = auto_field()

You can automatically generate fields for a model's columns using SQLAlchemyAutoSchema. The following schema classes are equivalent to the above.

from marshmallow_sqlalchemy import SQLAlchemyAutoSchema


class AuthorSchema(SQLAlchemyAutoSchema):
    class Meta:
        model = Author
        include_relationships = True
        load_instance = True


class BookSchema(SQLAlchemyAutoSchema):
    class Meta:
        model = Book
        include_fk = True
        load_instance = True

Make sure to declare Models before instantiating Schemas. Otherwise sqlalchemy.orm.configure_mappers() will run too soon and fail.

(De)serialize your data

author = Author(name="Chuck Paluhniuk")
author_schema = AuthorSchema()
book = Book(title="Fight Club", author=author)
session.add(author)
session.add(book)
session.commit()

dump_data = author_schema.dump(author)
print(dump_data)
# {'id': 1, 'name': 'Chuck Paluhniuk', 'books': [1]}

load_data = author_schema.load(dump_data, session=session)
print(load_data)
# <Author(name='Chuck Paluhniuk')>

Get it now

pip install -U marshmallow-sqlalchemy

Requires Python >= 3.7, marshmallow >= 3.0.0, and SQLAlchemy >= 1.3.0.

Documentation

Documentation is available at https://marshmallow-sqlalchemy.readthedocs.io/ .

Project Links

Homepage: https://marshmallow-sqlalchemy.readthedocs.io/


Download Details:

Author: marshmallow-code
Source Code: https://github.com/marshmallow-code/marshmallow-sqlalchemy

License: MIT license

#sqlalchemy #python 

Ruth  Gleason

Ruth Gleason

1667533980

SQLAlchemy Continuum: Versioning Extension for SQLAlchemy

SQLAlchemy-Continuum

Versioning and auditing extension for SQLAlchemy.

Features

  • Creates versions for inserts, deletes and updates
  • Does not store updates which don't change anything
  • Supports alembic migrations
  • Can revert objects data as well as all object relations at given transaction even if the object was deleted
  • Transactions can be queried afterwards using SQLAlchemy query syntax
  • Query for changed records at given transaction
  • Temporal relationship reflection. Version object's relationship show the parent objects relationships as they where in that point in time.
  • Supports native versioning for PostgreSQL database (trigger based versioning)

QuickStart

pip install SQLAlchemy-Continuum

In order to make your models versioned you need two things:

  1. Call make_versioned() before your models are defined.
  2. Add __versioned__ to all models you wish to add versioning to
from sqlalchemy_continuum import make_versioned


make_versioned(user_cls=None)


class Article(Base):
    __versioned__ = {}
    __tablename__ = 'article'

    id = sa.Column(sa.Integer, primary_key=True, autoincrement=True)
    name = sa.Column(sa.Unicode(255))
    content = sa.Column(sa.UnicodeText)


article = Article(name='Some article', content='Some content')
session.add(article)
session.commit()

# article has now one version stored in database
article.versions[0].name
# 'Some article'

article.name = 'Updated name'
session.commit()

article.versions[1].name
# 'Updated name'


# lets revert back to first version
article.versions[0].revert()

article.name
# 'Some article'

For completeness, below is a working example.

from sqlalchemy_continuum import make_versioned
from sqlalchemy import Column, Integer, Unicode, UnicodeText, create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import create_session, configure_mappers

make_versioned(user_cls=None)

Base = declarative_base()
class Article(Base):
    __versioned__ = {}
    __tablename__ = 'article'
    id = Column(Integer, primary_key=True, autoincrement=True)
    name = Column(Unicode(255))
    content = Column(UnicodeText)

configure_mappers()
engine = create_engine('sqlite://')
Base.metadata.create_all(engine)
session = create_session(bind=engine, autocommit=False)

article = Article(name=u'Some article', content=u'Some content')
session.add(article)
session.commit()
article.versions[0].name
article.name = u'Updated name'
session.commit()
article.versions[1].name
article.versions[0].revert()
article.name

Resources

http://i.imgur.com/UFaRx.gif

More information


Download Details:

Author: kvesteri
Source Code: https://github.com/kvesteri/sqlalchemy-continuum

License: BSD-3-Clause license

#sqlalchemy 

Dario  Schaden

Dario Schaden

1654071480

Pyramid SQLAlchemy: SQLAlchemy Integration for Pyramid

pyramid_sqlalchemy provides some basic glue to facilitate using SQLAlchemy with Pyramid.

SQLAlchemy relies on global state for a few things:

  • A MetaData instance which tracks all known SQL tables.
  • A base class for all models using the ORM.
  • A session factory.

Every application using SQLAlchemy must provides its own instance of these. This makes it hard create add-on packages that also use SQLAlchemy, since they either need to have their own SQLAlchemy state, which makes it hard to integrate them into your application, or they need to jump through multiple complex hoops to allow them share state with your application.

pyramid_sqlalchemy helps by providing a canonical location for the global SQLAlchemy state. In addition it provides a convenient way to configure SQLAlchemy in a Pyramid application.
from pyramid.config import Configurator
from pyramid_sqlalchemy import BaseObject

class MyModel(BaseObject):
    __tablename__ = 'my_model'
    ...

def main():
    config = Configurator()
    # Configure SQLAlchemy using settings from the .ini file
    config.include('pyramid_sqlalchemy')
    ...
    return config.make_wsgi_app()

Author: wichert
Source Code: https://github.com/wichert/pyramid_sqlalchemy
License: View license

#python #pyramid #sqlalchemy