The performance of GPT-4.5 on tacit knowledge and troubleshooting questions, compared to GPT-4, reflects advancements in handling complex and nuanced queries. While specific metrics for tacit knowledge and troubleshooting are not detailed in the available information, several key improvements in GPT-4.5 can be highlighted:
1. Instruction Hierarchy and Safety: GPT-4.5 has been trained to adhere to an instruction hierarchy, which helps mitigate the risk of prompt injections and other attacks. This means it is better at following system instructions over conflicting user messages, enhancing its ability to handle complex scenarios that might require troubleshooting[1]. In contrast, GPT-4 does not have this specific enhancement mentioned in the literature.
2. Ambiguous Questions: GPT-4.5 shows slightly lower accuracy on ambiguous questions compared to GPT-4o (a version of GPT-4), with an accuracy of 0.95 for GPT-4o versus 0.95 for GPT-4 and 0.95 for GPT-4.5. However, GPT-4.5's performance on unambiguous questions is similar to GPT-4o, indicating that both models handle clear queries effectively[1].
3. Tacit Knowledge and Troubleshooting: While specific metrics for tacit knowledge and troubleshooting are not provided, GPT-4.5's improved ability to handle conflicting messages and its enhanced instruction hierarchy suggest it might perform better in scenarios requiring nuanced understanding and troubleshooting. GPT-4, on the other hand, is known for its enhanced language comprehension and ability to provide more precise answers to complex queries[2].
4. Multimodal Capabilities: GPT-4, which is the base for GPT-4.5, offers significant improvements over GPT-3.5 in multimodal capabilities and handling complex queries. GPT-4 can process and respond to a wider range of data inputs, making it more versatile for tasks that might involve troubleshooting or tacit knowledge[2][6].
In summary, while GPT-4.5 builds upon the strengths of GPT-4, particularly in handling complex instructions and safety protocols, its specific performance on tacit knowledge and troubleshooting questions is not extensively detailed. However, its enhanced instruction hierarchy and improved handling of conflicting messages suggest it could be more effective in nuanced troubleshooting scenarios.
Citations:
[1] https://assets.ctfassets.net/kftzwdyauwt9/7EaDv6OaWHhXLAehUYu7Db/64e9f7916d3581ba4b5d0f0a6c5098d1/GPT-4-5_System_Card_2272025.pdf
[2] https://datasciencedojo.com/blog/gpt-3-5-vs-gpt-4-debate/
[3] https://www.opastpublishers.com/open-access-articles/evaluating-errors-and-improving-performance-of-chatgpt.pdf
[4] https://pmc.ncbi.nlm.nih.gov/articles/PMC11197181/
[5] https://community.openai.com/t/gpt-performance-is-not-follow-up-to-instructions-and-tasks/613298
[6] https://www.aubergine.co/insights/gpt-3-5-vs-gpt-4-an-in-depth-analysis-of-openais-language-models
[7] https://www.marketingaiinstitute.com/blog/the-ai-show-episode-137
[8] https://www.techtarget.com/searchenterpriseai/tip/GPT-35-vs-GPT-4-Biggest-differences-to-consider