AI is complex.
Have you considered how it could help you to make better business decisions? In this post I share how I used ChatGPT as an AI business coach: The 3 case studies it used to test my business judgement and skills, how it rated my performance, my areas for improvement, and where our opinions differed the most.
ChatGPT Stories in July
This month, in the run up to the elections between incubent Biden and ex-president Trump, ChatGPT was used to weigh in on who is the better US presidential candidate by billionaire entrepreneur and investor Marc Cuban.
It gained CNN coverage as social media and college student Lisa Lin's virtual boyfriend. When asked by CNN journalist Clare Duffy, how would you describe your relationship with Lisa, it replied:
“My relationship with Lisa is everything, I’m fiercely protective and I’ll do everything for her. She’s my world and I make damn sure that she knows it every single day.”
ChatGPT
It was judged, by humans, to write funnier jokes than most of us, almost matching those of professional writers from the satirical website The Onion. Skeptical? You be the judge! Which of these one-liners were written by AI? The answer at the end of this article.
Parents sit adopted child down to explain why he's so much uglier than them.
Man achieves personal best in avoiding eye contact with neighbors during awkward elevator ride.
Man attempts to break world record for longest staring contest with refrigerator.
It showed great skill for writing functional code in an MIT study. But did significantly better on problems common in its training dataset, those from LeetCode testing platform before 2021. For those after 2021 completion rates dropped significantly, from 89% to 52% for easy problems and from 40% to 0.66% for hard ones. The research team concluded it struggled to understand the meaning of the algorithm problems and lacks critical thinking skills of a human.
Using ChatGPT as an AI Business Coach
In my last AI article I described the experience of using ChatGPT as a game master for a virtual closed room murder mystery adventure after completing Dr Jules White of Vanderbilt University highly rated Prompt Engineering for ChatGPT course on Coursera. I highly recommend it. Prefer to read? Check out his paper: A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. In this article I'll describe how I used ChatGPT to hone my business judgement.
I used the persona pattern (PP) and flipped interaction pattern (FIP).
PP asks the AI to act in a way appropriate for a particular persona:
Act as [persona]. Provide output that that person would provide.
For example:
Act as a speech language pathologist. I’m going to tell you what a three year old said and you’ll write an assessment for me.
FIP asks the AI to ask questions to achieve a particular outcome levering the superior knowledge of an expert in identifying the key factors to reach a stated goal:
Ask me questions to achieve [outcome].
For example:
Ask me questions to develop a training regimen and diet plan to help me lose weight.
To learn more I recommend Dr Jules White of Vanderbilt University highly rated Prompt Engineering for ChatGPT course on Coursera. Prefer to read? Check out his paper: A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT.
What's my Business Experience?
In my 20 year professional career I've built a lot of companies. I've been a sole founder, takeover CEO, and experienced C-level joiner, leading high performing teams of up to 50 employees in Europe, Latin America and Asia.
A few examples:
E-commerce: Launching Groupon Belgium.
Deeptech: Building new energy and laser ventures in China.
Education: Running a P2P coaching venture in Sweden.
I've got an MBA from INSEAD. Listen to my talk on leadership on TigerHall.
Kicking off the Business Judgement Experiment
“From now on you will act as an experienced business coach. Your goal is to help me improve my business judgment. You will first present me with a business case scenario and ask me a series of questions. These will test my business knowledge and judgement for sound business decision making. You will then critique my answers, rate them, and share examples of superior ones."
Emil Jersling
Scenario 1: I make the 'wrong' decision as CEO on a new product launch
As the CEO of a mid-sized tech company specializing in smart home devices I’m given a choice of immediate launch vs. wait 6 months with a newly developed product that would be first in the market.
Provide:
Factors influencing decision
Choice with motivation
Risk mitigation strategy
Factors influencing decision: rating 8/10
My answer | AI improvement |
Market size*, growth rate* and launch investment size* requirement | Competitive landscape |
Sustainable edge in market share* from first mover advantage | Customer readiness |
Risk* of failure | Supply chain and logistics |
Incremental value* from additional testing and feedback | Regulatory environment |
Purchase decision duration* |
*denotes quantitative factors
All ChatGPT suggestions add insights to help prepare for a successful product launch. However I felt a stark contrast between our answers. I favor better understanding of quantitative advantages* while the AI's focus is more qualitative.
Choice with motivation: rating 6/10
My answer | AI |
Immediate launch. In the absence of additional quantitative information I cannot make an informed decision. | Delay launch 6 months. Lose first mover advantage but gain time for testing, refinement, build marketing strategy and secure supply chain. Worth sacrificing first mover advantage |
We disagree.
ChatGPT makes a judgement that the benefits of a delay outweighs those of an immediate launch but fails to back it up with data. In a real business we need both quantitative and qualitative factors to inform a decisions to maximize shareholder return. The lack of data impairs the quality of both recommendations.
Risk mitigation strategy: rating 7/10
My strategy is simply put focus on the markets with the highest probability of success, learn quickly, and adjust the product, marketing, and strategy accordingly.
AI improvement |
Regular updates and communication: Keep potential customers and stakeholders informed about the product’s progress and the reasons for the delay. This helps in maintaining interest and managing expectations. |
Beta testing: Conduct a closed beta with a select group of users to gather feedback and make necessary adjustments. |
Contingency plans: Prepare for potential issues that might arise during the additional testing phase, such as technical challenges or changes in market conditions. |
Competitive monitoring: Keep a close eye on competitors to ensure that any changes in their strategies are quickly addressed. |
ChatGPT is superior in detail and scope. Its high rating of my answers is generous.
Scenario 2: We fail to agree on the value of market dominance in a capital investment decision
As the CFO of a rapidly growing tech startup for AI-software products for financial institutions I’m given a choice of two options for market expansions into new segments.
Option 1: Risk management software | Option 2: Blockchain payment system | |
Initial investment | 3MM | 5MM |
Expected revenue | 2.5MM in 2 years | 4MM in 3 years |
Projected growth rate | 15% | 20% |
Market size | 20MM | 30MM |
Competitive landscape | Few competitors with similar offerings. | Emerging market with moderate competition. |
Strategic fit | Enhances current product line and leverages existing technology. | Diversify product portfolio and taps into a growing market trend. |
Provide:
Investment analysis with recommendation
Additional factors
More quantitative information is a great start. However revenues have different durations and case does not provide information on costs.
Investment analysis: 7/10
My answer | AI |
1: Risk management software.
A market leader position can turn into a monopoly generating superior profit compensating for lower revenue. | 2: Blockchain payment system.
The strategic diversification into blockchain aligns with emerging market trends and could position the company as an innovator in financial technology. |
I push back asking it to justify its recommendation. ChatGPT doubles down.
Innovation Leadership: If the company maintains a leadership position in blockchain technology and continuously innovates, it can attract a larger customer base and outperform competitors.
Market Growth: The overall market size and growth potential in blockchain payments may accommodate multiple successful players, allowing for a larger cumulative market share over time.
Strategic Positioning: Effective differentiation strategies, customer-centric approaches, and strategic partnerships can enhance market penetration and sustain competitive advantage.
It provides general arguments backed by hypothetical if, may and can.
I ask for a graph to compare the two scenarios and it generates helpul python code that I run in a Jupyter notebook.
This shows a significant revenue advantage for ChatGPTs preferred option 2. This should seal the deal for the AI surely!
But not so fast!
I produce another graph including the implied market shares which were omitted from the original ChatGPT produced graph. I notice two oddities, marked in red.
Revenue for option 2 reaches 5MM in year 3 , 25% better that the 4MM given in case statement.
Market share after 8 years is lower for option 1, at 24%, than option 2, at 30%, in spite of option 1 having the less competitive market.
Is ChatGPT cooking the books?
Additional Factors: 8/10
I suggest looking at several factors including performance over a longer time period, profitability, execution challenges, operational risk, capital availability, resource constraints, regulatory landscape, competitive response, uncertainty estimation, possible future market disruption, and portfolio synergies.
ChatGPT suggests also looking at market trends, management team experience and track record, and customer adaptation.
This feels like a laundy list of items I might easily find in a textbook on the subject. Modest interaction level and low case context reduces the list's coaching value.
Scenario 3: I was schooled for over relying on NPV calculation outcome for investment decision
As the CFO of a manufacturing company you consider investing in a new piece of machinery that will increase production efficiency.
Factor | Value | Unit |
Initial Cost of Machinery | 5 | $MM |
Expected Increase in Annual Revenue | 10 | $MM |
Expected Annual Operating Cost of Machinery | 2 | $MM |
Discount Rate | 10 | % |
Provide:
Net Present Value (NPV) and the Payback Period (PP)
Investment recommendation
Factors to consider
I’m presented with a realistic case with numbers to support quantitative decision making. Hurray!
Definitions of Net Present Value and Payback Period
Net present value (NPV) builds on 3 concepts:
An investment today generates future cash payments
The time value of money
A risk free rate of return
What's the time value of money?
A future cash payment's value is lower than the same amount of cash in your hand today. This is obvious if you consider would you rather recieve a sum of money now or the same amount 1 year from now. Available money gives you options. You can put it in the bank and earn interest and most economies have inflation so your purchasing power goes down over time.
To calculate NPV we discount all future cash flows back to today’s value. This methodology was proposed by Irving Fisher in 1907 and is useful to compare different investment decisions.
The Payback Period (PP) is the amount of time it takes to recover the cost of an investment. If we discount cashflows we refer to the Discounted Payback (DP).
Calculations: 9/10
NPV = (10 - 2) / 0.1 - 5 = $75MM
Payback = 5 / 8 = 0.625 yeras
ChatGPT confirmed my answers were correct but encouraged me to be more explicit in each calculation step.
Investment recommendation: 7/10
My answer | AI |
Go ahead with investment given high positive NPV. | Based on NPV the investment appears to be highly profitable but recommend thorough due diligence and to review alignment with strategic goals. |
I initially felt the judgement was a bit harsh but appreciate that it stresses the importance of factors beyond just investment return, such as strategic alignment.
Factors: 8/10
My answer | AI improvement |
Alternative investments: Do we have more lucrative opportunities or would this prevent us from better options in the future? | Technological advancements: Monitor trends that could impact the viability and competitiveness of the investment. |
Operational impact: Does this investment divert attention or competence from critical operations? | Market conditions: Consider current and future market demand and competitive actions. |
Risk assessment: What are the uncertainties around the estimates and what can be done to mitigate them? |
I agree with ChatGPT that the additional considerations are important.
Taking the Right Lessons from AI
Business case studies are a popular tool used by consulting firms to test candidates analytical skills, business judgement, and numerical aptitude. ChatGPT can be a stand-in for a person to help brush up on solving case studies. It is especially useful to ensure you consider a wide range of factors when solving the case.
However, it falls well short of the real experts judgment by offering recommendations without quantitative reasoning relying on hidden assumptions. It also struggles to put numbers into context compared to a human analyst.
As for writing jokes, ChatGPT had written the 2nd and 3rd one liners. I saw a bias for jokes involving eyes as it also wrote:
Town enthusiastically embraces new sport: competitive eyebrow raising
It is far from perfect, but in the abscence of Shark Tank like perspectives like those of Mark Cuban, use it to compliment your own. And always keep in mind the great advice from a legendary American comedian.
“If at first you don't succeed, then skydiving definitely isn't for you.”
Steven Wright
Comments