Being a part-time freelancer gets me a lot of questions from clients who are interested in exploring stuff they hear about from news sources and social media. They usually get the news of a new SOTA NLP model or a new advancement in an NLP model that makes performance on a task acceptable then they wonder, why not incorporate that to improve their business? To the extent, they might even create a specific business case around the new technology, not the other way around. My answer is usually to downscale the project into a Proof of Concept (PoC) with smaller well defined deliverables like an API and a deployment script and let them try it for their selves for a while before deciding on moving on with a bigger project.
This happens more often than not!
Because this is very common and happens all the time, I’ve decided to share my experience with both fellow freelancers and interested future clients. I’ll be writing a series of tutorials on some of the most requested NLP PoCs, things like Named Entity Recognition, Sentiment Analysis, and Machine Translation. I will keep this series open and without a predefined table of content. Whenever I find a new pattern, I’ll share and explain it here.
What to expect?
My plan is to write a short article explaining a certain PoC and the business cases around it while also explaining the engineering decisions behind it. I also plan on sharing a notebook or a Python project that has the solid implementation of said PoC. That way you can have a warm start that you can build from.
Most of the current PoCs will be a few lines of code thanks to the Huggingface transformers library. But sometimes we will need to write a little more if we want to customize a certain aspect or if we want to load data in a non-orthodox format. In those cases, I prefer using Pytorch with Pytorch Lightning that way we don’t have to write a lot of code.
The whole point of building a PoC is to be able to deliver ASAP a working prototype. That prototype should give your client what they expect. So that they can further investigate this from a business perspective and if it does align with their requirements, you’ll be able to build a system on top of it.
Summary
NLP Proof of Concepts is a very hot topic now. A lot of business managers are interested in exploring how NLP can benefit their business case. However, NLP projects tend to be long and expensive. In this article series, I’ll share and explain multi PoCs that I believe are requested these days. That way clients can easily deploy a PoC and explore it. Freelancers can also quickly customize these projects for their clients.