Dealing with PyTorch Custom Datasets

Dealing with PyTorch Custom Datasets

In this article, we are going to take a look at How to deal with Custom PyTorch Dataset. It is natural that we will develop our way of creating custom datasets while dealing with different Projects.

Custom datasets!! WHY??

Because you can shape it in a way you desire!!!

It is natural that we will develop our way of creating custom datasets while dealing with different Projects.

There are some official custom dataset examples on PyTorch Like here but it seemed a bit obscure to a beginner (like me, back then). The topics which we will discuss are as follows.

  1. Custom Dataset Fundamentals.
  2. Using Torchvision Transforms.
  3. Dealing with pandas (read_csv)
  4. Embedding Classes into File Names
  5. Using DataLoader

1. Custom Dataset Fundamentals.

A dataset must contain the following functions to be used by DataLoader later on.

  • __init__() function, the initial logic happens here, like reading a CSV, assigning transforms, filtering data, etc.,
  • __getitem__() returns the data and the labels.
  • __len__() returns the count of samples your dataset has.

Now, the first part is to create a dataset class:

from torch.utils.data.dataset import Dataset

class MyCustomDataset(Dataset):
    def __init__(self, ...):
        ## stuff

    def __getitem__(self, index):
        ## stuff
        return (img, label)

    def __len__(self):
        return count ## of how many examples(images?) you have

pytorch deep-learning dataloader machine-learning data-science deep learning

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

PyTorch for Deep Learning | Data Science | Machine Learning | Python

PyTorch for Deep Learning | Data Science | Machine Learning | Python. PyTorch is a library in Python which provides tools to build deep learning models. What python does for programming PyTorch does for deep learning. Python is a very flexible language for programming and just like python, the PyTorch library provides flexible tools for deep learning.

PyTorch for Deep Learning | Data Science | Machine Learning | Python

PyTorch is a library in Python which provides tools to build deep learning models. What python does for programming PyTorch does for deep learning.

Most popular Data Science and Machine Learning courses — July 2020

Most popular Data Science and Machine Learning courses — August 2020. This list was last updated in August 2020 — and will be updated regularly so as to keep it relevant

Data Augmentation in Deep Learning | Data Science | Machine Learning

Data Augmentation is a technique in Deep Learning which helps in adding value to our base dataset by adding the gathered information from various sources to improve the quality of data of an organisation.

Difference between Machine Learning, Data Science, AI, Deep Learning, and Statistics

In this article, I clarify the various roles of the data scientist, and how data science compares and overlaps with related fields such as machine learning, deep learning, AI, statistics, IoT, operations research, and applied mathematics.