The first and foremost challenge for data scientists in NLP tasks, after data cleaning, is to figure out how to represent their text mathematically. I was faced with this same challenge while working on building an open-domain semantic search based FAQ bot.
…
#tfidf-vectorizer #similarity-search #data-science #bert #chatbots #bert vs tf-idf embeddings in an enterprise chatbot