Python

Python

Python is an interpreted, high-level, general-purpose programming language. Python is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured (particularly, procedural), object-oriented, and functional programming. Python programming has various frameworks and features to expand in web application development, graphical user interfaces, data analysis, data visualization, machine learning, etc. Python programming language might not be an ideal choice for web application development, but is extensively used by many organizations for evaluating large datasets, for data visualization, for running data analysis or prototyping. Python programming language is gaining traction amongst users for data science whilst being outmoded as a web programming language.

Wsgitypes: MyPy Types For WSGI Applications

WSGI Types for Python

This is an attempt to bring some type safety to WSGI applications using Python's new typing features (TypedDicts, Protocols). It seems to work OK but may still be full of gaps, holes, bugs, missteps, etc. It helped bring a lot of extra safety to a couple of places that really needed it though, and seemed to remain quite stable for a couple of months.

This is implemented as a Python module, rather than MyPy stubs, as it represents a protocol things can satisfy rather than a set of types for something concrete.

This package came together during an exploration documented here: python/mypy#7654

Define a callable application as a class:

import wsgitypes

class MyApplication(wsgitypes.Application):
    def __call__(
        self,
        environ: wsgitypes.Environ,
        start_response: wsgitypes.StartResponse,
    ) -> wsgitypes.Response:
        my_header = environ.get("REQUEST_METHOD", "")
        return []

Environ should be type-safe:

class MyApplication(wsgitypes.Application):
    def __call__(
        self,
        environ: wsgitypes.Environ,
        start_response: wsgitypes.StartResponse,
    ) -> wsgitypes.Response:
        environ["wsgi.input"] # Good
        environ["wsgi.unpot"] # BORK! MyPy will catch this.
        return []

You can define your own extensions to Environ using TypedDict inheritance, like so:

class MyEnviron(wsgitypes.Environ):
    HTTP_X_MY_HEADER: t.Optional[str]

class MyApplication(wsgitypes.Application):
    def __call__(
        self,
        environ: MyEnviron,
        start_response: wsgitypes.StartResponse,
    ) -> wsgitypes.Response:
        environ = typing.cast(MyEnviron, environ)
        environ.get("HTTP_X_MY_HEADER") # Good
        return []

Note that you need to use typing.cast to convert the incoming Environ to your derived version. An attempt was made to use a type param for Environ, but it wasn't viable (even with GVR helping!): python/mypy#7654

Author: shabbyrobe
Source Code: https://github.com/shabbyrobe/wsgitypes
License: MIT license

#python 

Wsgitypes: MyPy Types For WSGI Applications
Willis  Mills

Willis Mills

1653278400

How to Iterate Or Loop Over Columns Of Python Pandas Data Frame

Python pandas tutorial on how to loop over columns or iterate over the columns using itermitems to get a value of single column or a specific row and apply any mathematical operation on that.

#dataframes #pandas #python 

How to Iterate Or Loop Over Columns Of Python Pandas Data Frame

Automate Twitter Processes with Python | Create Twitter Bot in Python

Create Twitter Bot with Python - TO BE REFILMED! XD

In todays tutorial we will learn how to automate Twitter processes, or in other words - we will create an extra friendly liking bot!  ๐Ÿค–๐Ÿค–๐Ÿค–
This live streaming tutorial is perfect for beginners! all you need is to have Python installed on your system (I'm using Python 3.9 in case you'd like to match versions ๐Ÿคช) and we will take care of everything else together!

Complete tutorial code can be found on my Github:
https://github.com/MariyaSha/TwitterBot 

The main purpose of this stream is to reflect on how easy it is to set up bots on social media platforms. Additionally, we will discuss whether you think it's good or bad - are extra layers of protection required? or do you like the current "wild wild west" situation? ๐Ÿ™ƒ
And even though our bot is going to be fluffy and harmless - can you imagine what experienced black hats with malicious intents have been doing with this technology throughout the years? (hint: phishing, impersonating and other fraud-related issues)

So I propose this experiment - will Twitter ever detect our bots? and how soon will this happen? (if at all)

#python #twitter #chatbot 

Automate Twitter Processes with Python | Create Twitter Bot in Python

ArcGIS Complete Tutorial For Beginners | Everything you need to get started with ArcGIS

This video included everything you need to get started with ArcGIS. We also added one real world GIS project like risk analysis in ArcGIS.

Table of Content

1.00 - where to download the software
1.30 - ArcGIS interface introduction
5.30 - Vector data vs raster data
8.00 - map template
10.00 - shape file creation
12.00 - map digitizing
23.00 - different ways of shape file editing
29.00 - label designing
36.00 - projection system change
37.00 - Area Calculation

40.00 - Saving as .MXD file ****************** Very important
43.00 - Geodatabase Design***************
01:2  - Topological Rule**********

1:11 - Study area mapping
1:34 - Different joins (merge, union, table join, excel sheet join)
1:56 - Different types of buffering
2:23 - Data processing (with little bit of SQL & Python commands)*********

2:37 - Raster data analysis 
        - Entire data frame clipping
        - Clipping
        - Extract by Mask
        - NDVI
        - IDW
        - zonal stat

2:56 -  Final Project
        - IDW
        - SRTM satellite data download (DEM)
        - slope derivation
         - Reclassification
         - Rating
         - Feature to raster
          -weighted sum
            - Final index

Dataset to Practice:
https://drive.google.com/drive/folders/1EwTVMmsrpYBK5ujmfeFAsmM4Wz2Who4s?usp=sharing

SUBSCRIBE: https://www.youtube.com/c/DataAnalyticsm/featured 

#arcgis #python 

ArcGIS Complete Tutorial For Beginners | Everything you need to get started with ArcGIS

์ž๋ฐ”์Šคํฌ๋ฆฝํŠธ ์ข…๋ง๊ฐ?! HTML์—์„œ ํŒŒ์ด์ฌ ์‹คํ–‰ํ•˜๋Š” PyScript ๋“ฑ์žฅ!

์ž๋ฐ”์Šคํฌ๋ฆฝํŠธ ์ข…๋ง๊ฐ?! HTML์—์„œ ํŒŒ์ด์ฌ ์‹คํ–‰ํ•˜๋Š” PyScript ๋“ฑ์žฅ!

End of JavaScript? Run Python with HTML

์š”์ฆ˜ ์™„์ ผ ๋‚œ๋ฆฌ๋‚œ 'ํŒŒ์ด์Šคํฌ๋ฆฝํŠธ' PyScript 10๋ถ„ ์„ค๋ช…!
์ž˜๊ฐ€~ ์ž๋ฐ”์Šคํฌ๋ฆฝํŠธ~ ์ด๋•Œ๊นŒ์ง€ ๊ณ ๋งˆ์› ๊ณ . ๋‹ค์‹  ๋ณด์ง€๋ง์ž (!?!?)
 

#pyscript #anaconda #python #javascript #html
 

์ž๋ฐ”์Šคํฌ๋ฆฝํŠธ ์ข…๋ง๊ฐ?! HTML์—์„œ ํŒŒ์ด์ฌ ์‹คํ–‰ํ•˜๋Š” PyScript ๋“ฑ์žฅ!

PyAutoGUI: A Cross-Platform GUI Automation Python Module

PyAutoGUI

PyAutoGUI is a cross-platform GUI automation Python module for human beings. Used to programmatically control the mouse & keyboard.

pip install pyautogui

Full documentation available at https://pyautogui.readthedocs.org

Simplified Chinese documentation available at https://github.com/asweigart/pyautogui/blob/master/docs/simplified-chinese.ipynb

Source code available at https://github.com/asweigart/pyautogui

If you need help installing Python, visit https://installpython3.com/

Dependencies

PyAutoGUI supports Python 2 and 3. If you are installing PyAutoGUI from PyPI using pip:

Windows has no dependencies. The Win32 extensions do not need to be installed.

macOS needs the pyobjc-core and pyobjc module installed (in that order).

Linux needs the python3-xlib (or python-xlib for Python 2) module installed.

Pillow needs to be installed, and on Linux you may need to install additional libraries to make sure Pillow's PNG/JPEG works correctly. See:

https://stackoverflow.com/questions/7648200/pip-install-pil-e-tickets-1-no-jpeg-png-support

http://ubuntuforums.org/showthread.php?t=1751455

If you want to do development and contribute to PyAutoGUI, you will need to install these modules from PyPI:

  • pyscreeze
  • pymsgbox
  • pytweening

Example Usage

Keyboard and Mouse Control

The x, y coordinates used by PyAutoGUI has the 0, 0 origin coordinates in the top left corner of the screen. The x coordinates increase going to the right (just as in mathematics) but the y coordinates increase going down (the opposite of mathematics). On a screen that is 1920 x 1080 pixels in size, coordinates 0, 0 are for the top left while 1919, 1079 is for the bottom right.

Currently, PyAutoGUI only works on the primary monitor. PyAutoGUI isn't reliable for the screen of a second monitor (the mouse functions may or may not work on multi-monitor setups depending on your operating system and version).

All keyboard presses done by PyAutoGUI are sent to the window that currently has focus, as if you had pressed the physical keyboard key.

    >>> import pyautogui
    >>> screenWidth, screenHeight = pyautogui.size() # Returns two integers, the width and height of the screen. (The primary monitor, in multi-monitor setups.)
    >>> currentMouseX, currentMouseY = pyautogui.position() # Returns two integers, the x and y of the mouse cursor's current position.
    >>> pyautogui.moveTo(100, 150) # Move the mouse to the x, y coordinates 100, 150.
    >>> pyautogui.click() # Click the mouse at its current location.
    >>> pyautogui.click(200, 220) # Click the mouse at the x, y coordinates 200, 220.
    >>> pyautogui.move(None, 10)  # Move mouse 10 pixels down, that is, move the mouse relative to its current position.
    >>> pyautogui.doubleClick() # Double click the mouse at the
    >>> pyautogui.moveTo(500, 500, duration=2, tween=pyautogui.easeInOutQuad) # Use tweening/easing function to move mouse over 2 seconds.
    >>> pyautogui.write('Hello world!', interval=0.25)  # Type with quarter-second pause in between each key.
    >>> pyautogui.press('esc') # Simulate pressing the Escape key.
    >>> pyautogui.keyDown('shift')
    >>> pyautogui.write(['left', 'left', 'left', 'left', 'left', 'left'])
    >>> pyautogui.keyUp('shift')
    >>> pyautogui.hotkey('ctrl', 'c')

Display Message Boxes

    >>> import pyautogui
    >>> pyautogui.alert('This is an alert box.')
    'OK'
    >>> pyautogui.confirm('Shall I proceed?')
    'Cancel'
    >>> pyautogui.confirm('Enter option.', buttons=['A', 'B', 'C'])
    'B'
    >>> pyautogui.prompt('What is your name?')
    'Al'
    >>> pyautogui.password('Enter password (text will be hidden)')
    'swordfish'

Screenshot Functions

(PyAutoGUI uses Pillow for image-related features.)

    >>> import pyautogui
    >>> im1 = pyautogui.screenshot()
    >>> im1.save('my_screenshot.png')
    >>> im2 = pyautogui.screenshot('my_screenshot2.png')

You can also locate where an image is on the screen:

    >>> import pyautogui
    >>> button7location = pyautogui.locateOnScreen('button.png') # returns (left, top, width, height) of matching region
    >>> button7location
    (1416, 562, 50, 41)
    >>> buttonx, buttony = pyautogui.center(button7location)
    >>> buttonx, buttony
    (1441, 582)
    >>> pyautogui.click(buttonx, buttony)  # clicks the center of where the button was found

The locateCenterOnScreen() function returns the center of this match region:

    >>> import pyautogui
    >>> buttonx, buttony = pyautogui.locateCenterOnScreen('button.png') # returns (x, y) of matching region
    >>> buttonx, buttony
    (1441, 582)
    >>> pyautogui.click(buttonx, buttony)  # clicks the center of where the button was found

How Does PyAutoGUI Work?

The three major operating systems (Windows, macOS, and Linux) each have different ways to programmatically control the mouse and keyboard. This can often involve confusing, obscure, and deeply technical details. The job of PyAutoGUI is to hide all of this complexity behind a simple API.

On Windows, PyAutoGUI accesses the Windows API (also called the WinAPI or win32 API) through the built-in ctypes module. The nicewin module at https://github.com/asweigart/nicewin provides a demonstration for how Windows API calls can be made through Python.

On macOS, PyAutoGUI uses the rubicon-objc module to access the Cocoa API.

On Linux, PyAutoGUI uses the Xlib module to access the X11 or X Window System.

Support

If you find this project helpful and would like to support its development, consider donating to its creator on Patreon.

Author: asweigart
Source Code: https://github.com/asweigart/pyautogui
License: BSD-3-Clause License

#python #gui 

PyAutoGUI: A Cross-Platform GUI Automation Python Module
Willis  Mills

Willis Mills

1653271140

How to Create & Process Hierarchical index on Pandas Series

Python pandas series tutorial on creating multiple index or hierarchical index and processing it by applying series functions.

#dataframes #pandas #python 

How to Create & Process Hierarchical index on Pandas Series
Willis  Mills

Willis Mills

1653268320

How to Rank a DataFrame in Python Pandas DataFrame Values

Python pandas tutorial for beginners on how to rank dataframe values in python using the rank function and re order the dataframe.

#dataframes #pandas #python 

How to Rank a DataFrame in Python Pandas DataFrame Values

Torchtyping: Type Annotations for A Tensor's Shape, Dtype, Names

torchtyping

Type annotations for a tensor's shape, dtype, names, ...

Turn this:

def batch_outer_product(x: torch.Tensor, y: torch.Tensor) -> torch.Tensor:
    # x has shape (batch, x_channels)
    # y has shape (batch, y_channels)
    # return has shape (batch, x_channels, y_channels)

    return x.unsqueeze(-1) * y.unsqueeze(-2)

into this:

def batch_outer_product(x:   TensorType["batch", "x_channels"],
                        y:   TensorType["batch", "y_channels"]
                        ) -> TensorType["batch", "x_channels", "y_channels"]:

    return x.unsqueeze(-1) * y.unsqueeze(-2)

with programmatic checking that the shape (dtype, ...) specification is met.

Bye-bye bugs! Say hello to enforced, clear documentation of your code.

If (like me) you find yourself littering your code with comments like # x has shape (batch, hidden_state) or statements like assert x.shape == y.shape , just to keep track of what shape everything is, then this is for you.


Installation

pip install torchtyping

Requires Python 3.7+ and PyTorch 1.7.0+.

Usage

torchtyping allows for type annotating:

  • shape: size, number of dimensions;
  • dtype (float, integer, etc.);
  • layout (dense, sparse);
  • names of dimensions as per named tensors;
  • arbitrary number of batch dimensions with ...;
  • ...plus anything else you like, as torchtyping is highly extensible.

If typeguard is (optionally) installed then at runtime the types can be checked to ensure that the tensors really are of the advertised shape, dtype, etc.

# EXAMPLE

from torch import rand
from torchtyping import TensorType, patch_typeguard
from typeguard import typechecked

patch_typeguard()  # use before @typechecked

@typechecked
def func(x: TensorType["batch"],
         y: TensorType["batch"]) -> TensorType["batch"]:
    return x + y

func(rand(3), rand(3))  # works
func(rand(3), rand(1))
# TypeError: Dimension 'batch' of inconsistent size. Got both 1 and 3.

typeguard also has an import hook that can be used to automatically test an entire module, without needing to manually add @typeguard.typechecked decorators.

If you're not using typeguard then torchtyping.patch_typeguard() can be omitted altogether, and torchtyping just used for documentation purposes. If you're not already using typeguard for your regular Python programming, then strongly consider using it. It's a great way to squash bugs. Both typeguard and torchtyping also integrate with pytest, so if you're concerned about any performance penalty then they can be enabled during tests only.

API

torchtyping.TensorType[shape, dtype, layout, details]

The core of the library.

Each of shape, dtype, layout, details are optional.

  • The shape argument can be any of:
    • An int: the dimension must be of exactly this size. If it is -1 then any size is allowed.
    • A str: the size of the dimension passed at runtime will be bound to this name, and all tensors checked that the sizes are consistent.
    • A ...: An arbitrary number of dimensions of any sizes.
    • A str: int pair (technically it's a slice), combining both str and int behaviour. (Just a str on its own is equivalent to str: -1.)
    • A str: str pair, in which case the size of the dimension passed at runtime will be bound to both names, and all dimensions with either name must have the same size. (Some people like to use this as a way to associate multiple names with a dimension, for extra documentation purposes.)
    • A str: ... pair, in which case the multiple dimensions corresponding to ... will be bound to the name specified by str, and again checked for consistency between arguments.
    • None, which when used in conjunction with is_named below, indicates a dimension that must not have a name in the sense of named tensors.
    • A None: int pair, combining both None and int behaviour. (Just a None on its own is equivalent to None: -1.)
    • A None: str pair, combining both None and str behaviour. (That is, it must not have a named dimension, but must be of a size consistent with other uses of the string.)
    • A typing.Any: Any size is allowed for this dimension (equivalent to -1).
    • Any tuple of the above. For example.TensorType["batch": ..., "length": 10, "channels", -1]. If you just want to specify the number of dimensions then use for example TensorType[-1, -1, -1] for a three-dimensional tensor.
  • The dtype argument can be any of:
    • torch.float32, torch.float64 etc.
    • int, bool, float, which are converted to their corresponding PyTorch types. float is specifically interpreted as torch.get_default_dtype(), which is usually float32.
  • The layout argument can be either torch.strided or torch.sparse_coo, for dense and sparse tensors respectively.
  • The details argument offers a way to pass an arbitrary number of additional flags that customise and extend torchtyping. Two flags are built-in by default. torchtyping.is_named causes the names of tensor dimensions to be checked, and torchtyping.is_float can be used to check that arbitrary floating point types are passed in. (Rather than just a specific one as with e.g. TensorType[torch.float32].) For discussion on how to customise torchtyping with your own details, see the further documentation.
  • Check multiple things at once by just putting them all together inside a single []. For example TensorType["batch": ..., "length", "channels", float, is_named].
torchtyping.patch_typeguard()

torchtyping integrates with typeguard to perform runtime type checking. torchtyping.patch_typeguard() should be called at the global level, and will patch typeguard to check TensorTypes.

This function is safe to run multiple times. (It does nothing after the first run).

  • If using @typeguard.typechecked, then torchtyping.patch_typeguard() should be called any time before using @typeguard.typechecked. For example you could call it at the start of each file using torchtyping.
  • If using typeguard.importhook.install_import_hook, then torchtyping.patch_typeguard() should be called any time before defining the functions you want checked. For example you could call torchtyping.patch_typeguard() just once, at the same time as the typeguard import hook. (The order of the hook and the patch doesn't matter.)
  • If you're not using typeguard then torchtyping.patch_typeguard() can be omitted altogether, and torchtyping just used for documentation purposes.
pytest --torchtyping-patch-typeguard

torchtyping offers a pytest plugin to automatically run torchtyping.patch_typeguard() before your tests. pytest will automatically discover the plugin, you just need to pass the --torchtyping-patch-typeguard flag to enable it. Packages can then be passed to typeguard as normal, either by using @typeguard.typechecked, typeguard's import hook, or the pytest flag --typeguard-packages="your_package_here".

Further documentation

See the further documentation for:

  • FAQ;
    • Including flake8 and mypy compatibility;
  • How to write custom extensions to torchtyping;
  • Resources and links to other libraries and materials on this topic;
  • More examples.

Author: patrick-kidger
Source Code: https://github.com/patrick-kidger/torchtyping
License: Apache-2.0 license

#python 

Torchtyping: Type Annotations for A Tensor's Shape, Dtype, Names
Dario  Schaden

Dario Schaden

1653264240

Pyramid LDAP: LDAP Authentication Policy for Pyramid

Pyramid LDAP

pyramid_ldap provides LDAP authentication services for your Pyramid application. Thanks to the ever-awesome SurveyMonkey for sponsoring the development of this package!

See the documentation at https://docs.pylonsproject.org/projects/pyramid_ldap/en/latest/ for more information.

This package will only work with Pyramid 1.3 and later.

Installation

pyramid_ldap uses pyldap which in turn requires libldap2 and libsasl2 development headers installed.

On Ubuntu 16.04 you can install them using the command apt-get install libldap2-dev libsasl2-dev.

Author: Pylons
Source Code: https://github.com/Pylons/pyramid_ldap
License: View license

#pyramid #python 

Pyramid LDAP: LDAP Authentication Policy for Pyramid

Locust: Scalable User Load Testing tool Written in Python

Locust

Locust is an easy to use, scriptable and scalable performance testing tool. You define the behaviour of your users in regular Python code, instead of being constrained by a UI or domain specific language that only pretends to be real code. This makes Locust infinitely expandable and very developer friendly.

To get started right away, head over to the documentation.

Features

Write user test scenarios in plain old Python

If you want your users to loop, perform some conditional behaviour or do some calculations, you just use the regular programming constructs provided by Python. Locust runs every user inside its own greenlet (a lightweight process/coroutine). This enables you to write your tests like normal (blocking) Python code instead of having to use callbacks or some other mechanism. Because your scenarios are โ€œjust pythonโ€ you can use your regular IDE, and version control your tests as regular code (as opposed to some other tools that use XML or binary formats)

from locust import HttpUser, task, between

class QuickstartUser(HttpUser):
    wait_time = between(1, 2)

    def on_start(self):
        self.client.post("/login", json={"username":"foo", "password":"bar"})

    @task
    def hello_world(self):
        self.client.get("/hello")
        self.client.get("/world")

    @task(3)
    def view_item(self):
        for item_id in range(10):
            self.client.get(f"/item?id={item_id}", name="/item")

Distributed & Scalable - supports hundreds of thousands of users

Locust makes it easy to run load tests distributed over multiple machines. It is event-based (using gevent), which makes it possible for a single process to handle many thousands concurrent users. While there may be other tools that are capable of doing more requests per second on a given hardware, the low overhead of each Locust user makes it very suitable for testing highly concurrent workloads.

Web-based UI

Locust has a user friendly web interface that shows the progress of your test in real-time. You can even change the load while the test is running. It can also be run without the UI, making it easy to use for CI/CD testing.

Locust UI charts Locust UI stats Locust UI workers Locust UI start test

Can test any system

Even though Locust primarily works with web sites/services, it can be used to test almost any system or protocol. Just write a client for what you want to test, or explore some created by the community.

Hackable

Locust's code base is intentionally kept small and doesn't solve everything out of the box. Instead, we try to make it easy to adapt to any situation you may come across, using regular Python code. If you want to send reporting data to that database & graphing system you like, wrap calls to a REST API to handle the particulars of your system or run a totally custom load pattern, there is nothing stopping you!

Links

Authors

License

Open source licensed under the MIT license (see LICENSE file for details).

Author: locustio
Source Code: https://github.com/locustio/locust
License: MIT License

#python 

Locust: Scalable User Load Testing tool Written in Python
Willis  Mills

Willis Mills

1653261060

How to Add or Subtract Two Different Series or DF in Python Pandas

Python pandas tutorial for beginners on how perform addition, substraction on two different series or dataframe on their numerical values.

#pandas #python #dataframe 

How to Add or Subtract Two Different Series or DF in Python Pandas

Tox: Aims To Automate and Standardize Testing in Python

tox automation project

Command line driven CI frontend and development task automation tool

At its core tox provides a convenient way to run arbitrary commands in isolated environments to serve as a single entry point for build, test and release activities.

tox is highly configurable and pluggable.

Example: run tests with Python 3.7 and Python 3.8

tox is mainly used as a command line tool and needs a tox.ini or a tool.tox section in pyproject.toml containing the configuration.

To test a simple project that has some tests, here is an example with a tox.ini in the root of the project:

[tox]
envlist = py37,py38

[testenv]
deps = pytest
commands = pytest
$ tox

[lots of output from what tox does]
[lots of output from commands that were run]

__________________ summary _________________
  py37: commands succeeded
  py38: commands succeeded
  congratulations :)

tox created two testenvs - one based on Python 3.7 and one based on Python 3.8, it installed pytest in them and ran the tests. The report at the end summarizes which testenvs have failed and which have succeeded.

Note: To learn more about what you can do with tox, have a look at the collection of examples in the documentation or existing projects using tox.

How it works

tox creates virtual environments for all configured so-called testenvs, it then installs the project and other necessary dependencies and runs the configured set of commands. See system overview for more details.

tox flow 

tox can be used for ...

  • creating development environments
  • running static code analysis and test tools
  • automating package builds
  • running tests against the package built by tox
  • checking that packages install correctly with different Python versions/interpreters
  • unifying Continuous Integration and command line based testing
  • building and deploying project documentation
  • releasing a package to PyPI or any other platform
  • limit: your imagination

Documentation

Documentation for tox can be found at Read The Docs.

Communication and questions

For the fastest and interactive feedback please join our server. If you have questions or suggestions you can first check if they have already been answered or discussed on our issue tracker. On Stack Overflow (tagged with tox).

Contributing

Contributions are welcome. See contributing and our Contributor Covenant Code of Conduct.

Currently, the code and the issues are hosted on GitHub.

The project is licensed under MIT.

Author: tox-dev
Source Code: https://github.com/tox-dev/tox
License: MIT License

#python 

Tox: Aims To Automate and Standardize Testing in Python
Willis  Mills

Willis Mills

1653253740

How to Identify & Drop Duplicate Values from Python Pandas DataFrame

Python tutorial for beginners on how to remove duplicate values from python pandas dataframe.


I have first shown the duplicated function of pandas which returns boolean value whether a particular row is duplicate or not.


After that I have sum method that you can chain with duplicated method to return the count' of rows that are duplicated in a dataset


After that I have shown how to find whether a particular column is having duplicate values or not as well as getting the count of duplicated values of a column.


By default duplicated pandas function keep the first row or value while identifying the duplicate value however I have shown you how can you keep the last value as an out of unique value dataset.


Finally I have shown the function to drop duplicate values out from the dataset and its various parameters like subset or keep etc. to further define the output of the unique values.

#python #pandas #dataframes 

How to Identify & Drop Duplicate Values from Python Pandas DataFrame

SQLAlchemy Stubs: Mypy Plugin and Stubs for SQLAlchemy

Mypy plugin and stubs for SQLAlchemy

This package contains type stubs and a mypy plugin to provide more precise static types and type inference for SQLAlchemy framework. SQLAlchemy uses some Python "magic" that makes having precise types for some code patterns problematic. This is why we need to accompany the stubs with mypy plugins. The final goal is to be able to get precise types for most common patterns. Currently, basic operations with models are supported. A simple example:

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String

Base = declarative_base()

class User(Base):
    __tablename__ = 'users'
    id = Column(Integer, primary_key=True)
    name = Column(String)

user = User(id=42, name=42)  # Error: Incompatible type for "name" of "User"
                             # (got "int", expected "Optional[str]")
user.id  # Inferred type is "int"
User.name  # Inferred type is "Column[Optional[str]]"

Some auto-generated attributes are added to models. Simple relationships are supported but require models to be imported:

from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from models.address import Address

...

class User(Base):
    __tablename__ = 'users'
    id = Column(Integer, primary_key=True)
    name = Column(String)
    address = relationship('Address')  # OK, mypy understands string references.

The next step is to support precise types for table definitions (e.g. inferring Column[Optional[str]] for users.c.name, currently it is just Column[Any]), and precise types for results of queries made using query() and select().

Installation

Install latest published version as:

pip install -U sqlalchemy-stubs

Important: you need to enable the plugin in your mypy config file:

[mypy]
plugins = sqlmypy

To install the development version of the package:

git clone https://github.com/dropbox/sqlalchemy-stubs
cd sqlalchemy-stubs
pip install -U .

Development Setup

First, clone the repo and cd into it, like in Installation, then:

git submodule update --init --recursive
pip install -r dev-requirements.txt

Then, to run the tests, simply:

pytest

Development status

The package is currently in alpha stage. See issue tracker for bugs and missing features. If you want to contribute, a good place to start is help-wanted label.

Currently, some basic use cases like inferring model field types are supported. The long term goal is to be able to infer types for more complex situations like correctly inferring columns in most compound queries.

External contributions to the project should be subject to Dropbox Contributor License Agreement (CLA).

Author: dropbox
Source Code: https://github.com/dropbox/sqlalchemy-stubs
License: Apache-2.0 license

#python #sqlalchemy 

SQLAlchemy Stubs: Mypy Plugin and Stubs for SQLAlchemy