Start with a simple model while labelled data accumulates. The host recommends beginning with something like logistic regression on a handful of hand-curated features, which also confirms whether the data collected so far carries any valuable signal. This is the third part of his three-part series on getting business value from machine learning.
Deep learning pays off only once data is plentiful. He says the threshold depends on the problem and its signal-to-noise ratio, but generally deep learning needs at least tens of thousands of data points to be meaningfully used, and some problems need millions or, in extreme cases, billions.
In production, speed usually matters more than accuracy. Users expect results in seconds or less, so if a large deep learning model adds only a few percent of ROC AUC over a regression but takes 10 seconds instead of one, the host argues the regression may be the better choice. In his view the accuracy gain may be imperceptible to users, but the wait will not be.