The math behind the OpenAI Jalapeño chip

OpenAI has developed the new Jalapeño chip in collaboration with Broadcom, a dedicated chip aimed at reducing infrastructure costs. The company incurred $8.4 billion last year to operate ChatGPT servers, with costs expected to rise to $14 billion this year as the number of users reaches 900 million weekly. The Jalapeño chip, OpenAI's first dedicated AI processor, enhances the performance of large language models rather than general tasks. It is designed to minimize data movement, thereby improving theoretical performance. While Nvidia achieves profit margins of up to 75%, OpenAI operates with a profit margin of only 33%.