Katrine Gorczany

Katrine Gorczany

1651450680

DoIT: Task Management & Automation Tool

doit - automation tool

doit comes from the idea of bringing the power of build-tools to execute any kind of task

doit can be uses as a simple Task Runner allowing you to easily define ad hoc tasks, helping you to organize all your project related tasks in an unified easy-to-use & discoverable way.

doit scales-up with an efficient execution model like a build-tool. doit creates a DAG (direct acyclic graph) and is able to cache task results. It ensures that only required tasks will be executed and in the correct order (aka incremental-builds).

The up-to-date check to cache task results is not restricted to looking for file modification on dependencies. Nor it requires "target" files. So it is also suitable to handle workflows not handled by traditional build-tools.

Tasks' dependencies and creation can be done dynamically during it is execution making it suitable to drive complex workflows and pipelines.

doit is build with a plugin architecture allowing extensible commands, custom output, storage backend and "task loader". It also provides an API allowing users to create new applications/tools leveraging doit functionality like a framework.

doit is a mature project being actively developed for more than 10 years. It includes several extras like: parallel execution, auto execution (watch for file changes), shell tab-completion, DAG visualisation, IPython integration, and more.

Sample Code

Define functions returning python dict with task's meta-data.

Snippet from tutorial:

def task_imports():
    """find imports from a python module"""
    for name, module in PKG_MODULES.by_name.items():
        yield {
            'name': name,
            'file_dep': [module.path],
            'actions': [(get_imports, (PKG_MODULES, module.path))],
        }

def task_dot():
    """generate a graphviz's dot graph from module imports"""
    return {
        'targets': ['requests.dot'],
        'actions': [module_to_dot],
        'getargs': {'imports': ('imports', 'modules')},
        'clean': True,
    }

def task_draw():
    """generate image from a dot file"""
    return {
        'file_dep': ['requests.dot'],
        'targets': ['requests.png'],
        'actions': ['dot -Tpng %(dependencies)s -o %(targets)s'],
        'clean': True,
    }

Run from terminal:

$ doit list
dot       generate a graphviz's dot graph from module imports
draw      generate image from a dot file
imports   find imports from a python module
$ doit
.  imports:requests.models
.  imports:requests.__init__
.  imports:requests.help
(...)
.  dot
.  draw

Project Details

license

The MIT License Copyright (c) 2008-2021 Eduardo Naufel Schettino

see LICENSE file

developers / contributors

see AUTHORS file

install

doit is tested on python 3.6 to 3.10.

The last version supporting python 2 is version 0.29.

$ pip install doit

dependencies

  • cloudpickle
  • pyinotify (linux)
  • macfsevents (mac)

Tools required for development:

  • git * VCS
  • py.test * unit-tests
  • coverage * code coverage
  • sphinx * doc tool
  • pyflakes * syntax checker
  • doit-py * helper to run dev tasks

development setup

The best way to setup an environment to develop doit itself is to create a virtualenv...

doit$ virtualenv dev
doit$ source dev/bin/activate

install doit as "editable", and add development dependencies from dev_requirements.txt:

(dev) doit$ pip install --editable .
(dev) doit$ pip install --requirement dev_requirements.txt

tests

Use py.test - http://pytest.org

$ py.test

documentation

doc folder contains ReST documentation based on Sphinx.

doc$ make html

They are the base for creating the website. The only difference is that the website includes analytics tracking. To create it (after installing doit):

$ doit website

spell checking

All documentation is spell checked using the task spell:

$ doit spell

It is a bit annoying that code snippets and names always fails the check, these words must be added into the file doc/dictionary.txt.

The spell checker currently uses hunspell, to install it on debian based systems install the hunspell package: apt-get install hunspell.

profiling

python -m cProfile -o output.pstats `which doit` list

gprof2dot -f pstats output.pstats | dot -Tpng -o output.png

releases

Update version number at:

  • doit/version.py
  • setup.py
  • doc/conf.py
  • doc/index.html
python setup.py sdist
python setup.py bdist_wheel
twine upload dist/doit-X.Y.Z.tar.gz
twine upload dist/doit-X.Y.Z-py3-none-any.whl

contributing

On github create pull requests using a named feature branch.

Financial contribution to support maintenance welcome.

Author: pydoit
Source Code: https://github.com/pydoit/doit
License: MIT License

#python 

What is GEEK

Buddha Community

DoIT: Task Management & Automation Tool
Origin Scale

Origin Scale

1620805745

Automation Management System

Want to try automated inventory management system for small businesses? Originscale automation software automate your data flow across orders, inventory, and purchasing. TRY FOR FREE

#automation #automation software #automated inventory management #automated inventory management system #automation management system #inventory automation

Stratus seo

Stratus seo

1625816471

Stratus: One of the best social media posting tools for efficient social media management

Efficient social media management could mean you getting the desired online recognition and leads for your business (if that was your intend to stay active on social media). Unfortunately, the common practice of social media management requires you to switch between multiple accounts of yours. This requires significant time and effort on your part. Stratus addresses this problem by bringing all of the social media channels on a single platform. You can access and manage your social media accounts in a single place while saving your time and effort. The user-friendly interface and advanced features integrated into the Stratus platform make it one of the best social media posting tools. To learn more or to sign up on Stratus, visit https://stratus.co/

#best social media posting tools #social media management #manage social media accounts in one place #best social media management tools #manage all social media in one place #social media management tools

Ian Robinson

Ian Robinson

1624399200

Top 10 Big Data Tools for Data Management and Analytics

Introduction to Big Data

What exactly is Big Data? Big Data is nothing but large and complex data sets, which can be both structured and unstructured. Its concept encompasses the infrastructures, technologies, and Big Data Tools created to manage this large amount of information.

To fulfill the need to achieve high-performance, Big Data Analytics tools play a vital role. Further, various Big Data tools and frameworks are responsible for retrieving meaningful information from a huge set of data.

List of Big Data Tools & Frameworks

The most important as well as popular Big Data Analytics Open Source Tools which are used in 2020 are as follows:

  1. Big Data Framework
  2. Data Storage Tools
  3. Data Visualization Tools
  4. Big Data Processing Tools
  5. Data Preprocessing Tools
  6. Data Wrangling Tools
  7. Big Data Testing Tools
  8. Data Governance Tools
  9. Security Management Tools
  10. Real-Time Data Streaming Tools

#big data engineering #top 10 big data tools for data management and analytics #big data tools for data management and analytics #tools for data management #analytics #top big data tools for data management and analytics

Aurelie Block

Aurelie Block

1598916060

Top 10 Automation Testing Tools: 2020 Edition

The demand for delivering quality software faster — or “Quality at Speed” — requires organizations to search for solutions in Agile, continuous integration (CI), and DevOps methodologies. Test automation is an essential part of these aspects. The latest World Quality Report 2018–2019 suggests that test automation is the biggest bottleneck to deliver “Quality at Speed,” as it is an enabler of successful Agile and DevOps adoption.

Test automation cannot be realized without good tools; as they determine how automation is performed and whether the benefits of automation can be delivered. Test automation tools is a crucial component in the DevOps toolchain. The current test automation trends have increased in applying artificial intelligence and machine learning (AI/ML) to offer advanced capabilities for test optimization, intelligent test generation, execution, and reporting. It will be worthwhile to understand which tools are best poised to take advantage of these trends.****

#automation-testing #automation-testing-tools #testing #testing-tools #selenium #open-source #test-automation #automated-testing

German Donnelly

German Donnelly

1592668860

Download Android SDK Manager and SDK Tools

In this tutorial, we’ll read about the Android SDK Manager. We will see what is SDK manager in Android and why and how it is important for Android. So, SDK stands for Software Development Kit, which is a collection of software tools required. SDK basically helps Android to download tools and recent versions of Android. Every time a new Android version is released, along with it is released an SDK corresponding to it. This SDK must be installed by the developers for the devices.
What is SDK Manager?
A Software development kit is a set of tools required for the development of applications for Android. It also ensures that the progress of App development goes as flat as pancakes. We need SDK irrespective of the language we are using. Android SDK comes wrapped up with the Android Studio these days. An Android SDK separates the tools, platforms and other components into packages. These can be downloaded from the SDK Manager.

#android tutorials #android sdk manager #android sdk manager download #android sdk tools #android studio sdk manager #sdk download #sdk manager #sdk tools