How to Save State in Docker — A Data Scientist’s Guide

In this post, we’re going to cover data persistence in Docker and how to get your images to interact with data outside of the container. If you’re familiar with Docker or have already walked through the previous posts in this series skip to the next heading!

So far in this series, we’ve covered how to get a basic Hello World! web app built using Flask and Python and deployed using Docker. In Part 2 we walked through an end-to-end machine learning example, building a Random Forest Classifier for the Iris dataset and serving it as a prediction web app using Flask, before sharing it on the public Docker Hub repository.

In case you missed them:

Docker for data scientists — Part 1

Docker for data scientists-Part 2

Now you may not want to share your images publically in all cases. Luckily there are many options available for you to build and maintain private repositories. Each of the major cloud providers Azure, AWS, and GCP have hosted repository options that are pretty straightforward to get up and running. For those of you keen to use Azure, I’ve also written a quick guide on how to get a private repository running using Docker and the Azure CLI here:

Private Docker repositories for data science with Azure

If you’re already up and running and just want to get going with data persistence, let’s get started.


What is GEEK

Buddha Community

How to Save State in Docker — A Data Scientist’s Guide
Sasha  Roberts

Sasha Roberts


Reform: Form Objects Decoupled From Models In Ruby


Form objects decoupled from your models.

Reform gives you a form object with validations and nested setup of models. It is completely framework-agnostic and doesn't care about your database.

Although reform can be used in any Ruby framework, it comes with Rails support, works with simple_form and other form gems, allows nesting forms to implement has_one and has_many relationships, can compose a form from multiple objects and gives you coercion.

Full Documentation

Reform is part of the Trailblazer framework. Full documentation is available on the project site.

Reform 2.2

Temporary note: Reform 2.2 does not automatically load Rails files anymore (e.g. ActiveModel::Validations). You need the reform-rails gem, see Installation.

Defining Forms

Forms are defined in separate classes. Often, these classes partially map to a model.

class AlbumForm < Reform::Form
  property :title
  validates :title, presence: true

Fields are declared using ::property. Validations work exactly as you know it from Rails or other frameworks. Note that validations no longer go into the model.


Forms have a ridiculously simple API with only a handful of public methods.

  1. #initialize always requires a model that the form represents.
  2. #validate(params) updates the form's fields with the input data (only the form, not the model) and then runs all validations. The return value is the boolean result of the validations.
  3. #errors returns validation messages in a classic ActiveModel style.
  4. #sync writes form data back to the model. This will only use setter methods on the model(s).
  5. #save (optional) will call #save on the model and nested models. Note that this implies a #sync call.
  6. #prepopulate! (optional) will run pre-population hooks to "fill out" your form before rendering.

In addition to the main API, forms expose accessors to the defined properties. This is used for rendering or manual operations.


In your controller or operation you create a form instance and pass in the models you want to work on.

class AlbumsController
  def new
    @form =

This will also work as an editing form with an existing album.

def edit
  @form =

Reform will read property values from the model in setup. In our example, the AlbumForm will call album.title to populate the title field.

Rendering Forms

Your @form is now ready to be rendered, either do it yourself or use something like Rails' #form_for, simple_form or formtastic.

= form_for @form do |f|
  = f.input :title

Nested forms and collections can be easily rendered with fields_for, etc. Note that you no longer pass the model to the form builder, but the Reform instance.

Optionally, you might want to use the #prepopulate! method to pre-populate fields and prepare the form for rendering.


After form submission, you need to validate the input.

class SongsController
  def create
    @form =

    #=> params: {song: {title: "Rio", length: "366"}}

    if @form.validate(params[:song])

The #validate method first updates the values of the form - the underlying model is still treated as immutuable and remains unchanged. It then runs all validations you provided in the form.

It's the only entry point for updating the form. This is per design, as separating writing and validation doesn't make sense for a form.

This allows rendering the form after validate with the data that has been submitted. However, don't get confused, the model's values are still the old, original values and are only changed after a #save or #sync operation.

Syncing Back

After validation, you have two choices: either call #save and let Reform sort out the rest. Or call #sync, which will write all the properties back to the model. In a nested form, this works recursively, of course.

It's then up to you what to do with the updated models - they're still unsaved.

Saving Forms

The easiest way to save the data is to call #save on the form.

if @form.validate(params[:song])  #=> populates album with incoming data
              #   by calling @form.album.title=.
  # handle validation errors.

This will sync the data to the model and then call

Sometimes, you need to do saving manually.

Default values

Reform allows default values to be provided for properties.

class AlbumForm < Reform::Form
  property :price_in_cents, default: 9_95

Saving Forms Manually

Calling #save with a block will provide a nested hash of the form's properties and values. This does not call #save on the models and allows you to implement the saving yourself.

The block parameter is a nested hash of the form input. do |hash|
    hash      #=> {title: "Greatest Hits"}

You can always access the form's model. This is helpful when you were using populators to set up objects when validating. do |hash|
    album = @form.model



Reform provides support for nested objects. Let's say the Album model keeps some associations.

class Album < ActiveRecord::Base
  has_one  :artist
  has_many :songs

The implementation details do not really matter here, as long as your album exposes readers and writes like Album#artist and Album#songs, this allows you to define nested forms.

class AlbumForm < Reform::Form
  property :title
  validates :title, presence: true

  property :artist do
    property :full_name
    validates :full_name, presence: true

  collection :songs do
    property :name

You can also reuse an existing form from elsewhere using :form.

property :artist, form: ArtistForm

Nested Setup

Reform will wrap defined nested objects in their own forms. This happens automatically when instantiating the form.

album.songs #=> [<Song name:"Run To The Hills">]

form =
form.songs[0] #=> <SongForm model: <Song name:"Run To The Hills">>
form.songs[0].name #=> "Run To The Hills"

Nested Rendering

When rendering a nested form you can use the form's readers to access the nested forms.

= text_field :title,         @form.title
= text_field "artist[name]",

Or use something like #fields_for in a Rails environment.

= form_for @form do |f|
  = f.text_field :title

  = f.fields_for :artist do |a|
    = a.text_field :name

Nested Processing

validate will assign values to the nested forms. sync and save work analogue to the non-nested form, just in a recursive way.

The block form of #save would give you the following data. do |nested|
  nested #=> {title:  "Greatest Hits",
         #    artist: {name: "Duran Duran"},
         #    songs: [{title: "Hungry Like The Wolf"},
         #            {title: "Last Chance On The Stairways"}]
         #   }

The manual saving with block is not encouraged. You should rather check the Disposable docs to find out how to implement your manual tweak with the official API.

Populating Forms

Very often, you need to give Reform some information how to create or find nested objects when validateing. This directive is called populator and documented here.


Add this line to your Gemfile:

gem "reform"

Reform works fine with Rails 3.1-5.0. However, inheritance of validations with ActiveModel::Validations is broken in Rails 3.2 and 4.0.

Since Reform 2.2, you have to add the reform-rails gem to your Gemfile to automatically load ActiveModel/Rails files.

gem "reform-rails"

Since Reform 2.0 you need to specify which validation backend you want to use (unless you're in a Rails environment where ActiveModel will be used).

To use ActiveModel (not recommended because very out-dated).

require "reform/form/active_model/validations"
Reform::Form.class_eval do
  include Reform::Form::ActiveModel::Validations

To use dry-validation (recommended).

require "reform/form/dry"
Reform::Form.class_eval do
  feature Reform::Form::Dry

Put this in an initializer or on top of your script.


Reform allows to map multiple models to one form. The complete documentation is here, however, this is how it works.

class AlbumForm < Reform::Form
  include Composition

  property :id,    on: :album
  property :title, on: :album
  property :songs, on: :cd
  property :cd_id, on: :cd, from: :id

When initializing a composition, you have to pass a hash that contains the composees. album, cd: CD.find(1))


Reform comes many more optional features, like hash fields, coercion, virtual fields, and so on. Check the full documentation here.

Reform is part of the Trailblazer project. Please buy my book to support the development and learn everything about Reform - there's two chapters dedicated to Reform!

Security And Strong_parameters

By explicitly defining the form layout using ::property there is no more need for protecting from unwanted input. strong_parameter or attr_accessible become obsolete. Reform will simply ignore undefined incoming parameters.

This is not Reform 1.x!

Temporary note: This is the README and API for Reform 2. On the public API, only a few tiny things have changed. Here are the Reform 1.2 docs.

Anyway, please upgrade and report problems and do not simply assume that we will magically find out what needs to get fixed. When in trouble, join us on Gitter.

Full documentation for Reform is available online, or support us and grab the Trailblazer book. There is an Upgrading Guide to help you migrate through versions.


Great thanks to Blake Education for giving us the freedom and time to develop this project in 2013 while working on their project.

Author: trailblazer
Source code:
License:  MIT license

#ruby  #ruby-on-rails

 iOS App Dev

iOS App Dev


Your Data Architecture: Simple Best Practices for Your Data Strategy

If you accumulate data on which you base your decision-making as an organization, you should probably think about your data architecture and possible best practices.

If you accumulate data on which you base your decision-making as an organization, you most probably need to think about your data architecture and consider possible best practices. Gaining a competitive edge, remaining customer-centric to the greatest extent possible, and streamlining processes to get on-the-button outcomes can all be traced back to an organization’s capacity to build a future-ready data architecture.

In what follows, we offer a short overview of the overarching capabilities of data architecture. These include user-centricity, elasticity, robustness, and the capacity to ensure the seamless flow of data at all times. Added to these are automation enablement, plus security and data governance considerations. These points from our checklist for what we perceive to be an anticipatory analytics ecosystem.

#big data #data science #big data analytics #data analysis #data architecture #data transformation #data platform #data strategy #cloud data platform #data acquisition

Java Questions

Java Questions


50 Data Science Jobs That Opened Just Last Week

Our latest survey report suggests that as the overall Data Science and Analytics market evolves to adapt to the constantly changing economic and business environments, data scientists and AI practitioners should be aware of the skills and tools that the broader community is working on. A good grip in these skills will further help data science enthusiasts to get the best jobs that various industries in their data science functions are offering.

In this article, we list down 50 latest job openings in data science that opened just last week.

(The jobs are sorted according to the years of experience r

1| Data Scientist at IBM

**Location: **Bangalore

Skills Required: Real-time anomaly detection solutions, NLP, text analytics, log analysis, cloud migration, AI planning, etc.

Apply here.

2| Associate Data Scientist at PayPal

**Location: **Chennai

Skills Required: Data mining experience in Python, R, H2O and/or SAS, cross-functional, highly complex data science projects, SQL or SQL-like tools, among others.

Apply here.

3| Data Scientist at Citrix

Location: Bangalore

Skills Required: Data modelling, database architecture, database design, database programming such as SQL, Python, etc., forecasting algorithms, cloud platforms, designing and developing ETL and ELT processes, etc.

Apply here.

4| Data Scientist at PayPal

**Location: **Bangalore

Skills Required: SQL and querying relational databases, statistical programming language (SAS, R, Python), data visualisation tool (Tableau, Qlikview), project management, etc.

Apply here.

5| Data Science at Accenture

**Location: **Bibinagar, Telangana

Skills Required: Data science frameworks Jupyter notebook, AWS Sagemaker, querying databases and using statistical computer languages: R, Python, SLQ, statistical and data mining techniques, distributed data/computing tools such as Map/Reduce, Flume, Drill, Hadoop, Hive, Spark, Gurobi, MySQL, among others.

#careers #data science #data science career #data science jobs #data science news #data scientist #data scientists #data scientists india

Gerhard  Brink

Gerhard Brink


Getting Started With Data Lakes

Frameworks for Efficient Enterprise Analytics

The opportunities big data offers also come with very real challenges that many organizations are facing today. Often, it’s finding the most cost-effective, scalable way to store and process boundless volumes of data in multiple formats that come from a growing number of sources. Then organizations need the analytical capabilities and flexibility to turn this data into insights that can meet their specific business objectives.

This Refcard dives into how a data lake helps tackle these challenges at both ends — from its enhanced architecture that’s designed for efficient data ingestion, storage, and management to its advanced analytics functionality and performance flexibility. You’ll also explore key benefits and common use cases.


As technology continues to evolve with new data sources, such as IoT sensors and social media churning out large volumes of data, there has never been a better time to discuss the possibilities and challenges of managing such data for varying analytical insights. In this Refcard, we dig deep into how data lakes solve the problem of storing and processing enormous amounts of data. While doing so, we also explore the benefits of data lakes, their use cases, and how they differ from data warehouses (DWHs).

This is a preview of the Getting Started With Data Lakes Refcard. To read the entire Refcard, please download the PDF from the link above.

#big data #data analytics #data analysis #business analytics #data warehouse #data storage #data lake #data lake architecture #data lake governance #data lake management

Ian  Robinson

Ian Robinson


Data Science: Advice for Aspiring Data Scientists | Experfy Insights

Around once a month, I get emailed by a student of some type asking how to get into Data Science, I’ve answered it enough that I decided to write it out here so I can link people to it. So if you’re one of those students, welcome!

I’ll segment this into basic advice, which can be found quite easily if you just google ‘how to get into data science’ and advice that is less common, but advice that I’ve found very useful over the years. I’ll start with the latter, and move on to basic advice. Obviously take this with a grain of salt as all advice comes with a bit of survivorship bias.

Less Basic Advice:

1. Find a solid community

2. Apply Data Science to Things you Enjoy

3. Minimize the ‘Clicks to Proof of Competence’

4. Learn Through Research or Entry Level Jobs

#big data & cloud #data science #data scientist #statistics #aspiring data scientist #advice for aspiring data scientists