Lightweight yet scalable TensorFlow workflow on Google Cloud

Lightweight yet scalable TensorFlow workflow on Google Cloud

My superpower toolkit: TFRecorder, TensorFlow Cloud, AI Platform Predictions and Weights & Biases. Lightweight yet scalable TensorFlow workflow on Google Cloud

I am just going to say it. I am absolutely overwhelmed and intimidated with the growing breadth and depth of machine learning (ML) today.

  • Need to build a high performant data pipeline? Learn Apache Beam or Spark and protocol buffers
  • Need to scale your model training? Learn AllReduce and multi-node distributed architectures
  • Need to deploy your models? Learn Kubernetes, TFServing, quantization, and API management
  • Need to track pipelines? Set up a metadata database, learn docker, and become a DevOps engineer

This does not even include the algorithm and modeling space which also makes me feel like an imposter without a research background. There has got to be an easier way!

I spent the last few weeks thinking about this dilemma and what I would recommend to a data scientist with a similar mindset as me. Many of those topics above are important to learn, especially if you want to focus on the new field of MLOps, but are there tools and technologies that can allow you to stand on the shoulder of giants?

Below are 4 such tools that abstract away much of the complexity and can allow you to more efficiently develop, track and scale your ML workflows.

  • TFRecorder** (via Dataflow)** : Turn data into TFRecords with ease by feeding in a CSV file. For images just provide the JPEG URIs and labels in the CSV. Scale to distributed servers with Dataflow without writing any Apache Beam code.
  • TensorFlow Cloud **(via AI Platform Training):** Scale your TensorFlow model training to single and multi-node clusters of GPUs on AI Platform Training with a simple API call.
  • AI Platform Predictions: Deploy your model as an API endpoint to a Kubernetes backed autoscaling service with GPUs, the same one used by Waze!
  • Weights & Biases: Log artifacts (datasets and models) to track versions and lineage across your development pipeline. Automatically generates a  tree of relationships between your experiments and artifacts.

tensorflow data-science google-cloud-platform machine-learning

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Train and Deploy TensorFlow Models using Google Cloud AI Platform

Train and Deploy TensorFlow Models using Google Cloud AI Platform. A practical workflow of TensorFlow model training and deploying

Top Google AI, Machine Learning Tools for Everyone

Google is much more than a search company. Learn about all the tools they are developing to help turn your ideas into reality through Google AI.

Model with TensorFlow and Serve on Google Cloud Platform

In this guide, we learn how to develop a TensorFlow model and serve it on the Google Cloud Platform (GCP). We consider a regression ...

Most popular Data Science and Machine Learning courses — July 2020

Most popular Data Science and Machine Learning courses — August 2020. This list was last updated in August 2020 — and will be updated regularly so as to keep it relevant

How SAP customers benefit from Google Cloud analytics & machine learning

Google Cloud’s data analytics and AI and machine learning solutions can help SAP customers store, analyze, and derive insights from all their data in the cloud. You can shift your SAP applications to the cloud to take full advantage of a flexible, scalable solution that eliminates ongoing infrastructure maintenance costs; leverage BigQuery for your enterprise data to unlock new business value; integrate machine learning into business processes; or mix and match solutions to suit your needs now and in the future.