I am just going to say it. I am absolutely overwhelmed and intimidated with the growing breadth and depth of machine learning (ML) today.

• Need to build a high performant data pipeline? Learn Apache Beam or Spark and protocol buffers
• Need to scale your model training? Learn AllReduce and multi-node distributed architectures
• Need to deploy your models? Learn Kubernetes, TFServing, quantization, and API management
• Need to track pipelines? Set up a metadata database, learn docker, and become a DevOps engineer

This does not even include the algorithm and modeling space which also makes me feel like an imposter without a research background. There has got to be an easier way!

I spent the last few weeks thinking about this dilemma and what I would recommend to a data scientist with a similar mindset as me. Many of those topics above are important to learn, especially if you want to focus on the new field of MLOps, but are there tools and technologies that can allow you to stand on the shoulder of giants?

Below are 4 such tools that abstract away much of the complexity and can allow you to more efficiently develop, track and scale your ML workflows.

• TFRecorder** (via Dataflow)** : Turn data into TFRecords with ease by feeding in a CSV file. For images just provide the JPEG URIs and labels in the CSV. Scale to distributed servers with Dataflow without writing any Apache Beam code.
• **TensorFlow Cloud **(via AI Platform Training): Scale your TensorFlow model training to single and multi-node clusters of GPUs on AI Platform Training with a simple API call.
• AI Platform Predictions: Deploy your model as an API endpoint to a Kubernetes backed autoscaling service with GPUs, the same one used by Waze!
• Weights & Biases: Log artifacts (datasets and models) to track versions and lineage across your development pipeline. Automatically generates a  tree of relationships between your experiments and artifacts.

#tensorflow #data-science #google-cloud-platform #machine-learning