Deep Research and Knowledge Value

Key Takeaways

deep research and knowledge value was published on Monday February 10th 2025 when did you feel the AGI this is a question that been floating around AI circles for a while and it's a hard one to answer for two reasons first what is Agi and second feel is a bit like obscenity as Supreme Court Justice Potter Stewart famously said in jaab Bellis V Ohio I know it when I see it I gave my definition of AGI in ai's uneven arrival quote what o03 and inference time scaling point to is something different AI that can actually be given tasks entrusted to complete them this by extension looks a lot more like an independent worker than an assistant ammunition rather than a rifle sight that may seem an OD analogy but it comes from a talk Keith R boy gave at Stanford my definition of AGI is that it can be ammunition i. it can be given a task entrusted to complete it at a good enough rate my Definition of artificial super intelligence ASI is the ability to come up with the tasks in the first place end quote the feel part of that question is a more recent discovery deep research from open aai feels like AGI I just got a new employee for the shockingly low price of $200 a month deep research bullets open aai announced deep research in a February 2nd blog post today we're launching deep research in Chachi PT a new agentic capability that conducts multi-step research on the internet for complex tasks it accom accomplishes in tens of minutes what will take a human many hours deep research is open ai's next agent that can do work for you independently you give it a prompt and Chachi PT will find analyze and synthesize hundreds of online sources to create a comprehensive report at the level of a research analyst powered by a version of the upcoming open aai 03 model that's optimized for web browsing and data analysis it leverages reasoning to search interpret and analyze massive amounts of text images and PDFs on the internet pivoting as needed in reaction to information it encounters the ability to synthesize knowledge is a prerequisite for creating new knowledge for this reason deep research marks a significant step toward our broader goal of developing AGI which we have long envisioned as capable of producing novel scientific research it's honestly hard to keep track of open eyes AGI definitions these days CEO Sam Altman just yesterday defined it as a system that can tackle increasingly complex problems at human level in many fields but in my rather more modest definition deep research sits right in the middle of that excerpt it synthesizes research in an economically valuable way but doesn't create new knowledge I already published two examples of deep research in last Tuesday's update while I suggest reading the whole thing to summarize first I published my brief review of Apple's recent earnings including three observations it was notable that Apple earned record Revenue even though iPhone sales were down year-over-year in the latest data point about the company's transformation into a Services Juggernaut China sales were down again but this wasn't a new trend it actually goes back nearly a decade but you can only see that if you realize how the Huawei chip band gave Apple a temporary boost in the country third while Apple Executives claimed that Apple intelligence drove iPhone sales there really wasn't any evidence in the geographic sales number supporting that assertion second I published a deep research report using a generic prompt I am Ben Thompson the author of straty this is important information because I want you to understand my previous analysis of apple and the voice in which I write on Strater I want a research report about Apple's latest earnings in the style and voice of Strater that is in line with my previous analysis third I published a deep research report using a prompt that incorporated my takeaways from the earnings I am Ben Thompson the author of straty this is important information because I want you to understand my previous analysis of apple and the voice in which I write on Strat I want a research report about Apple's latest earnings for fiscal year 2025 q1 calendar year 2024 Q4 there are a couple of angles I am particularly interested in first there is the overall trend of services Revenue carrying the company's earnings how is that Trend continued what does it mean for margins Etc second I am interested in the China angle my theory is that Apple's recent decline in China is not new but is actually part of a longer Trend going back nearly a decade I believe that Trend was arrested by the chip ban on Huawei but that that was only a temporary bump in terms of a long-term decline in addition I would like to marry this to deeper analysis of the Chinese phone market the distinction between first tier cities and the rest of China and what that says about Apple's prospects in the country third what takeaways are there about Apple's AI prospects the company claims that Apple intelligence is helping sales in markets where it has launched but isn't this a function of not being available in China please deliver this report in a format and style that is suitable for straty you can read the update for the output but this was my evaluation quote the first answer was decent given the pity of instruction it's really more of a summary than anything but there are a few insightful points the second answer was considerably more impressive the answer relied much more heavily on my previous posts and weave points I've made in the past into the answer I don't to be honest think I learned anything new but I think that anyone encountering this topic for the first time would have or to put it another way were I looking for a research assistant I would consider hiring whoever wrote the second answer end quote in other words deep research isn't a rifle barrel but for this question at least it was a pretty decent piece of ammunition deep research examples still that ammunition wasn't that valuable to me I I read the transcript of Apple's earnings call before my 8 a.m. dithering recording and came up with my three points immediately that's the luxury of having thought about and covered Apple for going on 12 years and as I noted above the entire reason that the second deep research report was interesting was because I came up with the ideas and deep research substantiated them the substantiation however wasn't nearly to the standard in my very bias subjective opinion of aery update I found a much more beneficial use case the next day before I conduct aery interview I do several hours of research on the person I'm interviewing their professional background the company they worked for Etc in this case I was talking to Bill McDermot the chairman and CEO of service now a company I am somewhat familiar with but not intimately so so I asked deep research for help I'm am going to conduct an interview with Bill McDermot the CEO of service now and I need to do research about both mcdermit and service now to prepare my questions first I want to know more about MC dmid and his background I ideally there are some good profiles of him I can read I know he used to work at sap and I would like to know what is relevant about his experience there also how and why did he take the service now job then what is the background of service now how did it get started what was its initial product Market fit and how has it expanded over time what kind of companies you service now what is the service now business model what is its go-to Market strategy MC government wants to talk about service now's opportunities in AI what are those opportunities and how are they meaningfully unique or different from simple automation what do users think of service now is it very ugly and hard to use why is it very sticky what attracts companies to it what competitors does service now have can it be a platform for other companies or is there an opportunity to disrupt service now what other questions do you have that would be useful for me to ask you can use previous trary interviews as a resource to understand the kinds of questions I typically asks I found the results eminently useful although the questions were pretty mid I did spend some time doing additional reading of things like earnings reports before conducting the interview with my own questions in short it saved me a fair bit of time and gave me a place to start from and that alone more than paid for my monthly subscription another compelling example came in researching a friend's complicated medical issue I'm not going to share my prompt in results for obvious reasons what I will note is that this friend has been struggling with this issue for over a year and has seen multiple doctors and tried several different remedies deep research identified a possible issue in 10 minutes that my friend has only just learned about from a specialist last week well it is still to be determined if this is the answer he is looking for it is notable that deep research may have accomplished in 10 minutes what has taken my friend many hours over many months with many medical professionals it is the final example however that is the most interesting precisely because it is the question on which deep research most egregiously failed I generated a report about another friend's industry asking for the major players supply chain analysis customer segments Etc it was by far my most comprehensive and detailed prompt and sure enough deep research came back with a fully fleshed out report answering all of my questions it was also completely wrong but in a really surprising way the best way to characterize the issu is go back to that famous Donald Rumsfeld quote there are known knowns there are things we know we know we also know there are known unknowns that is to say we know there are some things we do not know but there are also unknown unknowns the ones we don't know we don't know the issue with the report I generated and once again I'm not going to share the results but this time for reasons that are not obvious is that it completely missed a major entity in the industry in question this particular entity is not a well-known brand but is a major player in the supply chain it is a significant enough entity that any report about the industry that did not include them is if you want to be generous incomplete it is in fact the fourth categorization that Rumsfeld didn't mention the Unknown Known anyone who read the report that deep research generated would be given the illusion of knowledge but would not know what they think they know knowledge value what of the most painful lessons of the internet was the realization by Publishers that news was worthless I'm not speaking about societal value but rather economic value something everyone knows is both important and also non- monetizable which is to say that the act of publishing is economically destructive I wrote in Publishers and the pursuit of the past quote too many newspaper Advocates utterly and completely failed to understand this the truth is that newspapers made money in the past not by providing societal value but by having quasy monopolistic control of print advertising in their geographic area the societal value was a bonus thus when shav complains that quote today's internet Distribution Systems distort the flow of economic value derived from good reporting end quote he is in fact conflating societal value with economic value the latter does not exist and has never existed this failure to understand the past leads to a misdiagnosis of the present Google and Facebook are not profitable because they took newspapers reporting they are profitable because they took their advertising moreover the utility of both platforms is so great that even if all newspaper content were magically removed which has been tried in Europe the only thing that would change is that said newspapers would lose even more Revenue as they lost traffic this is why this solution is so misplaced newspapers no longer have a monopoly on Advertising can never compete with the internet when it comes to bundling content and news remains both valuable to society and for the same reasons worthless economically reaching lots of people is inversely correlated to extracting value and facts both real and fake ones spread for free end quote it is maybe a bit extreme to say has always been such in truth it is very very hard to draw direct lines from the analog era defined as it was by friction and scarcity to the internet era's transparency and abundance it may have technically been the case that those of us old enough to remember news stands bought the morning paper because a local light manufacturing company owned printing presses delivery trucks and an advertising sales team but we too believed we simply wanted to know what was happening now we get that need fulfilled for free and probably by social media for better or worse I sometimes wish I knew less still what deep research reveals is how much more could be known I read a lot of things on the internet but it's not as if I will ever come close to reading everything moreover as the amount of slop increases whether human or AI generated the difficulty in finding the right stuff to read is only increasing this is also one problem with deep research that is worth pointing out the worst results are often paradoxically for the most popular topics precisely because those are the topics that are the most likely to be contaminated by slop the more precise and obscure the topic the more more likely it is that deep research will have to find papers and articles that actually cover the topic well this graph however is only half complete as the example of my friend's industry shows there's a good chance that deep research particularly as it evolves will become the most effective search engine there has ever been it will find whatever information there is to find about a particular topic and present it in a relevant way it is the death in other words of security through obscurity previously we shifted from a world where you had to pay for the news to the new news being fed to you now we will shift from a world where you had to spend hours researching a topic to having a topic reported to you on command unless of course the information that matters is not on the internet this is why I'm not sharing the Deep research report that provoked this Insight I happen to know something about the industry in question which is not related to Tech to be clear because I have a friend who works in it and it is suddenly clear to me how much future economic value is wrapped up in information not being public in this case the entity in question is privately held so there aren't stock market filings public reports barely even a web page and so AI is blind there is another example this time in Tech of just how valuable secrecy can be Amazon launched S3 the first primitive offered by AWS in 2006 followed by ec2 later that year and soon transformed startups and Venture Capital what wasn't clear was to what extent AWS was transforming Amazon the company slowly transitioned amazon.com to AWS and that was reasonable enough to list aws's financials under amazon.com until 2012 and then under other along with things like credit card and then small amounts of advertising Revenue after that the grand Revelation would come in 2015 when Amazon announced in January that it would break AWS out into a separate division for reporting purposes from a reuter's report at the time after years of giving investors the cold shoulder amazon.com Inc is starting to warm up to Wall Street the number one Us online retailer was unused usually forthcoming during its fourth quarter earnings call on Thursday saying it will break out results this year for the first time for its fast growing cloud computing unit Amazon web services the additional information shared during Amazon's fourth quarter results as well as its emphasis on becoming more efficient signal a new willingness by Amazon Executives to listen to investors as well quote this quarter Amazon flexed its muscles and said this is what we can do when we focus on profits end quot said Rob Plaza senior Equity analyst for key Private Bank quote if they could deliver that upper teen's low 20s Revenue growth and be able to deliver profits on top of that the stock is going to respond end quote the change is unlikely to be dramatic when asked whether this quarter marked a permanent shift in Amazon's relationship with Wall Street Plaza laughed quote I wouldn't be chasing the stock hair based on that end quot still the shift is a good sign for investors who have been clamoring for Amazon to disclose more about its fastest growing and likely most profitable division that some analysts say accounts for 4% of total sales in fact AWS accounted for nearly 7% of total sales and it was dramatically more profitable than anyone expected the Revelation caused such a massive uptick in the stock price that I called it the AWS IPO quote one of the technology industry's biggest and most important IPOs occurred late last month with a valuation of 2 5.6 billion that's more than Google which ipoed at a valuation of $24.6 billion and certainly a lot more than Amazon which finished its first day on the public markets with a valuation of $438 million don't feel too bad for the ladder though the IPO I'm talking about was Amazon web services and it just so happens to still be owned by the same e-commerce company that went public nearly 20 years ago I'm obviously being factious there was no actual IPO for AWS just an additional line item on Amazon's Financial reports finally breaking out the cloud computing service Amazon pioneered 9 years ago that line item though was almost certainly the primary factor in driving an overnight increase in Amazon's market capitalization from $182 billion in April 23rd to $27 billion in April 24th it's not only that AWS is a strong offering and a growing Market with impressive economics it also May in the end be the key to realizing the potential of amazon.com itself end quote that 25 5.6 billion increase in market cap however came with its own costs both Microsoft and Google doubled down on their own cloud businesses in response and while AWS is still the market leader it faces stiff competition that's a win for consumers and customers but also a reminder that known unknowns have a value all their own surfacing data I wouldn't go so far as to say that Amazon was wrong to disclose aws's financials in fact SEC rules would have required as much once AWS revenues became 10% % of the company's overall business to date is 15% which might seem low until you remember that Amazon's Topline Revenue includes first-party e-commerce sales moreover releasing aws's financials gave investors renewed confidence in the company giving management freedom to continue investing heavily in capital expenditures for both AWS and the e-commerce business fueling Amazon's transformation into a logistics company the point rather is to note that secrets are valuable what is interesting to consider is what this means for AI tools like deep research hedge funds have long known the value of proprietary data paying for everything from satellite images to traffic observers and everything in between in order to get a market Edge my suspicion is that work like this is going to become even more valuable as security by obscurity disappears it's going to be more difficult to harvest Alpha from Reading endless Financial filings when an AI can do that research in a fraction of the time the problem with those hedge fund reports however is that they themselves are proprietary however they are not a complete Secret After All the way to monetize that research is through making trades on the open market which is to say those reports have an impact on prices prices are a signal that is available to everyone and it's going to become an increasingly important one that by extension is why AI like deep research are one of the most powerful arguments yet for prediction markets prediction markets had their own moment in the sun last fall during the US presidential election when they were far more optimistic about a trump Victory than plls however the potential in fact the the necessity of prediction markets is only going to increase with AI ai's capability of knowing everything that is public is going to increase the incentive to keep things secret prediction markets and everything will provide a profit incentive for knowledge to be disseminated by price if nothing else it is also interesting that prediction markets have become associated with crypto another technology that is poised to come into its own in an AI dominated World infinite content generation increases the value of digital scarcity in verification just as infinite transparency increases the value of secrecy AI is likely to be the key to tying all this together a combination of verifiable information and understandable price movements may be the only way to derive any meaning from the slop that is slowly drowning the internet this is the other reality of AI and why it is inescapable just as the internet's transparency and freedom to publish has devolved into torrent of information of questionable veracity requiring ever more heroic efforts to parse and undeniable opportunities to thrive by building independ Brands like this site AI will both be the cause of further pollution of the information ecosystem and simultaneously the only way out deep research impacts much of this is in the not so distant future for now deep research is one of the best bargains of Technology yes $200 a month is a lot and yes deep research is limited by the quality of information on the internet and is highly dependent on the quality of the prompt I can't say that I've encountered any particular Sparks of creativity at least in Arenas that I know well but at the same time there is a lot of work that isn't creative in nature but necessary all the same I personally feel much more productive and truth be told I was never going to hire a researcher anyways that though speaks to the Peril in two distinct ways first one reason I've never hired a researcher is that I see tremendous value in the search for and sifting of information there is so much you learn on the way to a destination and I value that learning will Serendipity be an unwelcome casualty to reports on demand moreover what of those who haven't to take the above example been reading Apple earnings reports for 12 years or thinking in reading about technology for three decades what will be lost for the next generation of analysts and of course there is the job question lots of other entities employ researchers in all sorts of fields and those celeries are going to be increasingly hard to justify I've known intellectually that AI would replace wide swalls of knowledge work it is another thing to fill it viscerally at the same time that is why the value of secrecy is worth calling out secrecy is its own form of friction the purposeful imposition of scarcity on valuable knowledge it speaks to what will be valuable in an AI denominated future yes the real world and human denominated Industries will rise in economic value but so will the tools and infrastructure that both Drive original research and discoveries and the mechanisms to price it the power of AI at least on our current trajectory comes from knowing everything the perhaps doomed response of many will be to build walls tollgates and marketplaces to produ and harvest the fruits of their human Expeditions for more analysis like this please like And subscribe and visit cher.com and listen to the sharp Tech podcast also check out the asianometry channel on YouTube to learn more about the technology changing our world