Summary Nathan Labenz on AI pricing - Marginal REVOLUTION marginalrevolution.com
6,587 words - html page - View html page
One Line
AI services are becoming more affordable due to competition and cost factors, and it is being used to replace headcount jobs, disrupt global businesses, and suppress facts, with prices expected to continue to fall in the coming years.
Key Points
- Marginal Revolution 2023 is a project to keep up with the latest wave of digital technology, and its implications include how humans compare to nature and that this technology may soon leave us behind.
- AI services are decreasing in cost due to competition, cost of training, and the ability to use a particular platform.
- AI search can disrupt Google's search/adwords business model, and HTML made the first major advance in what became the world wide web.
- AI technology has the potential to disrupt top global businesses like Google Search, but IP law reform is needed to reduce the power of tech barons.
- Gary Marcus is skeptical about the current version of AI, believing the singular approach of generative AI will also disappoint.
- Distributed training techniques, such as on consumer devices or reusing compute, and distillation, mixture of experts and pruning for sparsity can reduce compute costs.
Summaries
251 word summary
AI services are becoming more affordable due to competition and cost factors, and AI is being used to replace headcount jobs in a variety of industries. Marginal Revolution 2023 has identified implications of digital technology, such as the hack and leak of a proprietary neural net model and NovelAI's success in enhancing the model. Despite limitations, AI has the potential to disrupt top global businesses like Google Search. Costs are falling due to improvements such as sparser matrices and reinforcement learning, but IP law reform is needed to reduce the power of tech barons.
ChatGPT was tested with summarizing a text but failed, showing it cannot be trusted for such tasks. Tyler Cowen and Alexander Tabarrok's project, Marginal Revolution 2023, has identified implications of digital technology, such as the hack and leak of a proprietary neural net model and NovelAI's success in enhancing the model. Cowen's comparison of generative AI to crypto is inaccurate as the price of generative AI is falling. Gary Marcus explains this is due to its unreliability and lack of truthfulness. Natacha de Mahieu's Theatre of Authenticity explores the link between tourism and spectacle.
AI is also being used to suppress facts, allowing governments to control what can be discussed. This problem is likely to worsen as machines take over the task. Distributed training and cost-reduction techniques such as 8-bit and 4-bit inference, Open Instruct, RLHF, and distillation are being used to reduce AI compute costs, with prices expected to continue to fall in the coming years.
488 word summary
Distributed training and cost-reduction techniques such as 8-bit and 4-bit inference, Open Instruct, RLHF, and distillation are being used to reduce AI compute costs. OpenAI, StabilityAI, and others have already achieved significant cost reductions. Prices are expected to continue to fall in the coming years. AI is also being used to suppress facts, allowing governments to control what can be discussed. This problem is likely to worsen as machines take over the task. However, not everyone is willing to fix it. The limbo after a mistrial is limited by the statute of limitations. Innocent until proven guilty is undermined by the inaccuracy of AI models. I experienced the power of food to evoke memories while trying ChatGPT, which gave correct responses for some questions but failed to provide an "exceedingly eloquent" description of a hot dog experience. This reminded me of the nuances between guilty, innocent, and no longer facing trial without a determination of guilt or innocence. Cowen's comparison of generative AI to crypto is inaccurate as the price of generative AI is falling. Gary Marcus explains this is due to its unreliability and lack of truthfulness, which has led to the rise of AI-created fake vacations on TikTok. Natacha de Mahieu's Theatre of Authenticity explores the link between tourism and spectacle. AI can now make pictures for a low cost, but ChatGPT may be able to do the marketing and contractual work needed. Early success of Facebook and PayPal was due to being early and having a network effect. AI is likely to be oversold, but prices will fall, and it may be useful in enhancing productivity.
Despite its limitations, AI technology has the potential to disrupt top global businesses like Google Search. Costs are falling due to improvements such as sparser matrices and reinforcement learning, but IP law reform is needed to reduce the power of tech barons. AI chat can be transformative in some cases, but its success will depend on how useful it is to people. HTML made the first major advance in what became the world wide web, but ChatGPT has a long way to go to reach that level of change. Google's search/adwords business model may be disrupted by AI search, but paid advertising embedded in the results could ruin the technology. Mosaic was the real game-changer, but self-driving cars and ChatGPT may face the same obstacle. My attitude is still wait and see. AI services are becoming more affordable due to competition and cost factors. Business models are being developed to cover fixed costs, and AI is being used to replace headcount jobs like scheduling and medical imaging analysis. ChatGPT was tested with summarizing a text but failed, showing it cannot be trusted for such tasks. Tyler Cowen and Alexander Tabarrok's project, Marginal Revolution 2023, has identified implications of digital technology, such as the hack and leak of a proprietary neural net model and NovelAI's success in enhancing the model.
1316 word summary
Marginal Revolution 2023 is Tyler Cowen & Alexander Tabarrok's project to keep up with the latest wave of digital technology. They are adept at catching the wave, and have identified some of the implications of this technology. These implications include how profoundly stupid humans are compared to nature and that this technology may soon leave us in the dust. There has already been a hack and leak of a proprietary neural net model, which has become the basis of a thriving enthusiast community. NovelAI continues to print money by enhancing the model and providing a web interface. The thousandth post on I asked GPT to do x and it gave mey was about story grammar and ChatGPT. The post included a short story about Princess Aurora that the AI had created in an earlier session, and a request to retell the story with XP-708-DQ as the protagonist. Another suggestion was to give it a story from the Middle Ages and ask it to rewrite it in the vernacular of today.
ChatGPT was also tested with summarizing Sir Gawain and the Green Knight, which it botched. This showed that it cannot be trusted for tasks such as summarizing long texts.
The discussion also included AI replacing headcount jobs, such as scheduling for a trucking firm and medical imaging analysis. Costs of creating the program are fixed, while marginal costs are low, making it an attractive revenue opportunity. Prices of AI services are decreasing due to competition and other factors, such as cost of training and the ability to use a particular platform. This decrease in cost is seen in online gambling programs and sites, where customer acquisition costs are high but customers tend to stick around. AI services could become like cloud providers, providing basic infrastructure including hardware and charging subscription fees. Business models for firms in the industry are being developed to cover fixed costs. Google's search/adwords business model may be disrupted by AI search. If the AI answer is useful, few people click through to the links. AI based services already use a free service with limited usage and require payment for professional use. Paid advertising embedded in the results could ruin the technology, as it has with search. Mosaic was the real game-changer, as it was obviously so right from the start. ChatGPT gets us partway to decent machine-generated text, but self-driving cars cannot yet handle the big wide world of driving and ChatGPT may face the same obstacle. My attitude is still wait and see. ChatGPT has a long way to go to reach the immense change made by HTML, which allowed Google to organize its search index. Gopher did not develop as competition to Alta Vista, a search machine that in several ways remains better than today's Google. Using the internet in 1995 was like playing with ChatGPT now, but HTML made the first major advance in what became the world wide web. Project Gutenberg got started in 1971, and was never hard to search. There is skepticism about what AI can do today, similar to the people circa 1995 saying "What ever are people going to use the Internet for?" Reliable truth is not what defines the usefulness of AI models, and it is important to ensure that good is good enough and does not hurt the development and benefits of AI. Marcus highlighted the limitations of current AI models, as well as a possible path towards Artificial General Intelligence (AGI). Despite these limitations, AI technology has the potential to disrupt top global businesses like Google Search. Costs are falling due to improvements such as sparser matrices and reinforcement learning, but IP law reform is needed to reduce the power of tech barons. AI chat can be transformative in some cases, but its success will depend on how useful it is to people. ChatGPT may be able to do the marketing and contractual work needed, but the value of both would be reduced due to fragmentation. Early success of Facebook and PayPal was due to being early and having a network effect. AI is likely to be oversold, but prices will fall, and it may be useful in enhancing productivity. Gary Marcus is skeptical about the current version of AI, which is why autonomous cars have been disappointing. He believes the singular approach of generative AI will also disappoint. Cowen's comparison of generative AI to crypto as the next big thing is inaccurate as the price of generative AI is falling, while the price of crypto was rising. Gary Marcus responds that the low price of generative AI is due to its unreliability and lack of truthfulness. AI can now create fake vacations through TikTok, with pictures, text, and video tailored to the user. Natacha de Mahieu's photography project, Theatre of Authenticity, explores the link between tourism and spectacle, and how people act when they travel, particularly for social media. With the extra geolocation package, users can make it appear as if they were at trendy spots before the crowds to preserve their status online. AI can now make pictures for a low cost, which some may find eliminates the need for a real vacation. I held the hot dog aloft, savoring the nostalgia it evoked. Memories of childhood summers spent with family flooded my mind. I was struck by the power of food to unlock such a depth of emotion and recollection. Then I tried ChatGPT, which gave a correct conceptual summary of my research area, a similar summary when asked about an algorithm, and an incorrect answer when asked a math question. It answered questions about history and literature with Wikipedia-like responses. Finally, I asked for an "exceedingly eloquent" description of a hot dog experience and received something written at a middle school level. This experience reminded me of the power of memory and the importance of understanding the nuances between guilty, innocent, and no longer facing trial without a determination of guilt or innocence. The limbo after a mistrial is limited, but nonetheless exists, and is covered by the statute of limitations. In the American legal system, you are not declared innocent after a hung jury or mistrial, and the DA can still retry within a reasonable timeframe. The two binary settings of guilty and innocent are complemented by a non-binary justice system. A factual failure that is highly polished may be taken as true by many, which undermines the concept of innocent until proven guilty. Questions such as whether innocent and exonerated are the same should not be asked of ChatGPT. AI is increasingly being used to suppress facts, allowing governments to control what can and cannot be discussed. This can be seen in Twitter's suppression of facts, which was done by a small group of people at the direction of the government. This problem will be further exacerbated as machines take over the job of suppressing facts, allowing the government to quickly and efficiently shut down debate. AI engineers may not agree with this, but their opinions will be irrelevant. AI models can be used to tell people whatever the government wants them to believe, such as false information about laptops or the 2016 election. However, not everyone wants this fixed, as evidenced by ChatGPT's refusal to answer certain questions. The hallucination problem is more transient than many think and teams are working on solutions. Distributed training techniques, such as on consumer devices or reusing compute, and distillation, mixture of experts and pruning for sparsity can reduce compute costs. 8-bit and 4-bit inference, Open Instruct, RLHF and other techniques are being used to reduce costs even further. OpenAI has reduced embeddings prices by 99.8%, StabilityAI has reduced prices on Stable Diffusion to $0.002 / image, and OpenAI has reduced core LLM pricing by 2/3rds. It is expected that prices will continue to fall over the next couple of years, approaching marginal costs for common use cases.