Knowledge is power. Knowledge is important in AI because it takes knowledge to effectively deploy AI solutions, with the emphasis on the word effectively. Let me digress with a brief anecdote to make the point:
My front yard and my neighbor’s front yard both looked bad - bald patches, weeds, and what grass there was looked sick. We went two very different directions. I called a company that specializes in lawn care, handed them my credit card, and now my grass looks great. I have little understanding of what they did, but it worked.
In contrast, my neighbor read everything he could about lawn care, talked to people at the local agricultural extension service, laid out a meticulous plan, and set it in motion. He explained the research to me, the next steps, and the timeline. I saw that play over the course of the next couple of months. Now both of our lawns look great!
Note the tradeoffs, whether it be for lawn care or deploying AI.
You can pay in either money or time, but you’ll have to pay. Once you’ve acquired knowledge, it can be reused. If I had a second house and ruined the lawn there, I’d have to hire someone again; my neighbor could just redeploy his knowledge. But there are degrees of knowledge, and depth takes lots of time. Getting really good at something takes time, effort, and practice. So, if you have the time to wait and resources to invest in learning, learning how to deploy AI may or may not be the best option for you.
As an AI consultant who has been working with AI professionally for more than 30 years, I’ve seen lots of variations of how companies are approaching AI:
So what are the dangers of building AI systems without the help of a true expert, with hard won knowledge and experience?
Wasted time: Expertise allows one to look at a problem, quickly see the right solution, and easily deal with setbacks while implementing it. I’ve talked to many potential clients who hired an AI “expert” who was learning while doing. Learning while doing is fine if you’ve got someone inside the company who will be able to redeploy their acquired knowledge. But learning while doing is slow, with time spent searching Stack Overflow, talking to ChatGPT, watching YouTube videos to try to understand what the temperature parameter does in a large language model (LLM), and so on. And when your AI system makes mistakes on new customer data, the process repeats. How do we make it better? Let’s watch a video or ask ChatGPT.
On the other hand, experts get you from problem description to problem solution as fast as possible, which is incredibly important when your competitors are racing with you to launch their AI solution or your customers are demanding to know when you are deploying AI.
Wasted money: Time is money, so if you waste time, you’re wasting money. But it’s not as simple as that. Modern AI solutions, such as those involving LLMs, are data and compute hungry, and different LLMs have different capabilities at different price points. One approach is to pay for access to the most powerful LLM on the market and use it for every part of your system.
But, at Synaptiq, we’ve seen cases where generating answers to customer facing questions needs a high-end model, but determining if those questions are on topic does not. For example, your customer support chatbot might use the latest OpenAI model to generate the best possible answers, but a cheap and local open source model is probably good enough to check that the question is about customer support and not asking for a brownie recipe. And that local model is going to be faster and cheaper than an OpenAI call. Without an expert, you’ll spend more on developing AI solutions.
Sub-optimal solutions: It’s not unusual for someone to see all of the buzz around deep learning and LLMs and assume that they are the best solution for their business problem. But what’s best can involve lots of factors, such as ease of maintenance, understandability, long-term costs, and performance (e.g., accuracy of predictions). Understanding what those factors are, how they play against each other, and how the AI-driven approach to the problem affects them is not easy for someone who is not an AI expert.
What is easy is picking an approach and letting the chips fall where they may, which is what a novice does out of necessity. After doing that a few dozen times you get better at knowing, as opposed to searching for, the right solution tailored to the problem at hand. Without an expert, your AI solution will not be as useful.
The changing AI landscape: There are few fields that change as quickly as AI. ChatGPT 3.5 came out in November of 2022 and shocked the world. That was followed quickly by versions 4o and o1, each significantly better than its predecessor.
That applies in all areas of AI. A paper is published announcing a new algorithm for, say, object detection and tracking that beats all competitors. Three months later a new paper appears and the old king is dethroned. Knowledge of the field is essential to understand whether the new algorithm’s extra 2 points in accuracy is worth the computational complexity, whether it’s robust enough in the target domain, or if it will require an idiosyncratic infrastructure that will make it hard to swap in newer algorithms as they appear.
Hidden gotchas: Have you ever spun up a powerful computer in AWS to grind on a big machine learning problem, forgot it was running, and a week later spent almost $1000 dollars on that big machine sitting there doing nothing after it finished running your program. I did. Once.
When building AI solutions there are lots of hidden gotchas like that, waiting to pounce on the newbie. Here are a few:
Because AI systems are largely data driven and data is typically collected by people (with the help of computer programs), it is often the case that the data embody human biases and the resulting AI systems do as well. For example, a dataset on in-game purchases in Call of Duty will be dominated by males because they account for about 80% of players. If you use that dataset to market in-game purchases to women, it won’t do well without taking special steps that an expert knows about, like building gender-specific models, or rebalancing the data, using cost-sensitive methods that increase the weight on female players.
A related problem is that the behavior of users of an AI system may be different from the data used to develop it. Amazon uses AI to distill reviews down into facets, or elements of a product that users like or dislike. If all of the reviews are about cameras, then “heavy” is a bad thing, but if reviews about cast iron skillets seep in then “heavy” could be a good thing. The issue of “data drift” can cause the performance of AI systems to degrade through time without you noticing, unless you have an AI expert who knows to look for those kinds of things.
Data volume and quality tend to be the factors that are most important to determining the success of an AI project. Everyone knows that dirty data can be cleaned, but limited data can also be expanded. You can create new data by making small random changes to your existing data, add to your data with similar data from open sources, apply domain knowledge to increase the power of AI models so they can squeeze more information from it, and so on. Without an expert, you may think your data is not good enough or there’s not enough of it, but an expert can help you get the best results possible from limited data resources.
There are many paths to success with AI. You can learn as you go, get there a little slower with a system that has a few warts, but you’ll be able to apply what you learned in the future and get better through time. You can get an expert on board and get there faster with an elegant and powerful solution. I hope that this article has helped you to think about which of these paths, or some combination of them, is best for your organization.
---
Dr. Tim Oates is the co-founder & Chief Data Scientist at Synaptiq. He has 20+ years of experience in guiding 100+ organizations of all sizes in applying artificial intelligence and machine learning to business problems.
Want to learn more from Dr. Tim Oates? Subscribe to Synaptiq's monthly newsletter at the bottom of this page.
Want to chat with Synaptiq about your AI or data challenges?
Image by Wolfgang Hasselmann on Unsplash
Synaptiq is an AI and data science consultancy based in Portland, Oregon. We collaborate with our clients to develop human-centered products and solutions. We uphold a strong commitment to ethics and innovation.
Contact us if you have a problem to solve, a process to refine, or a question to ask. You can learn more about our story through our past projects, blog, or podcast.