Tensorflow: Logits and labels must have the same first dimension

I’m new to machine learning in TF. I have this dataset which I generated and exported into a .csv file. It is here: tftest.csv.

The ‘distributions’ column corresponds to a unique system of equations which I have tried to condense down into a series of digits in SageMath. The ‘probs’ column correspond to whether one should mutiply a given equation by a given monomial of the equation, based on the row and column it is located in. The above is just for overview and is not related to my actual question.

Anyways, here’s my code. I’ve tried to explain it as best as I can with annotations.

import csv
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
import tensorflow.keras as keras

distribution_train = []
probs_train = []
# x_train = []
# y_train = []

with open('tftest.csv') as csv_file:
    csv_reader = csv.reader(csv_file, delimiter=',')

    for row in csv_reader:
        distribution_train.append(row[0])
        probs_train.append(row[1])

'''
Get rid of the titles in the csv file
'''
distribution_train.pop(0)
probs_train.pop(0)

'''
For some reason everything in my csv file is stored as strings.
The below function is to convert it into floats so that TF can work with it.
'''
def num_converter_flatten(csv_list):
    f = []
    for j in range(len(csv_list)):
        append_this = []
        for i in csv_list[j]:
            if i == '1' or i == '2' or i == '3' or i == '4' or i == '5' or i == '6' or i == '7' or i == '8' or i =='9' or i =='0':
                append_this.append(float(i))
        f.append((append_this))

    return f

x_train = num_converter_flatten(distribution_train)
y_train = num_converter_flatten(probs_train)

x_train = tf.keras.utils.normalize(x_train, axis=1)
y_train = tf.keras.utils.normalize(y_train, axis=1)

model = tf.keras.models.Sequential()

model.add(tf.keras.layers.Flatten())

model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu))
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu))

'''
I'm making the final layer 80 because I want TF to output the size of the
'probs' list in the csv file
'''

model.add(tf.keras.layers.Dense(80, activation=tf.nn.softmax))

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=5)

However, when I run my code, I get the following error.

tensorflow.python.framework.errors_impl.<span class="hljs-symbol">InvalidArgumentError:</span> logits <span class="hljs-keyword">and</span> labels must have the same first dimension, got logits shape [<span class="hljs-number">32</span>,<span class="hljs-number">80</span>] <span class="hljs-keyword">and</span> labels shape [<span class="hljs-number">2560</span>]
 [[{{node loss/output_1_loss/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWi

I searched online for this error, but I can’t seem to understand why it’s cropping up. Can anyone help me understand what’s wrong with my code? If there are any questions as well, please leave a comment and I’ll do my best to answer them.

#python #tensorflow #machine-learning