Hôm nay mình vừa đọc một bài viết tổng hợp lại insights/observations của 1 bác khi so sánh ChatGPT với Claude. Khá hay nên share lên cho mọi người =)) bình thường mình ngại share mấy bài về AI lắm vì đa số toàn nói chung chung kiểu “something big is happening”, mà tác giả này mình theo dõi một thời gian thấy có substance nên là mạn phép share lên.
Cơ bản là bác này dump 5 năm dữ liệu marketing vô 2 con LLM và so sánh kết quả.
Takeaway:
- Claude tốt hơn ChatGPT khá nhiều trong chuyện phân tích dữ liệu marketing, nó tìm ra seasonality, sự thay đổi thuật toán của platforms, etc. Còn GPT thì nói chung chung.
- Domain expertise cải thiện chất lượng output của LLM rất đáng kể. Cái này cũng không có gì bất ngờ, nhưng mà có vài ví dụ về apply domain expertise vào prompt mình thấy useful.
Once the data was cleaned of pseudo-duplicate columns and the LLM was given additional instructions (what to look for and what to ignore), it produced useful insights. Here’s an example of such prompting:
“What about a correlation between ‘Cost Per Result’ with metrics like ‘Frequency’, ‘Adds to Cart’, ‘Checkouts Initiated’? Nothing on the website or product pricing has changed since these ads started, leading me to believe that recent performance drops are related to something in the advertising performance data. My goal is to identify leading indicators of performance and monitor them when implementing a new ad strategy.”
- Nguyên cái đoạn này:
Frankly, good “prompting” isn’t too different from providing good requirements and instructions to a human team. The same principles apply: provide necessary context, details, expected outcomes, and any other critical operational information. In that sense, a great template is – to some degree – a distillation of operational expertise.
- LLM hữu ích nhất trong hai tác vụ - đó là so sánh đối chiếu và validate intuition.
- Comparative Performance
Claude and ChatGPT produced similar and complementary outputs. They identified broadly similar personas while suggesting different campaign ideas. Both gave better marketing recommendations than the agencies I’d been speaking with—despite agencies having access to the same data (my website) to work with.- Validation of Intuition
LLM recommendations aligned closely with what my partner and I intuitively knew and had been planning to work on. This occurred even when I reviewed my prompts for potential bias (none found). As an extremely analytical person who doesn’t always trust his gut feelings, this was healthy reinforcement that I was on the right track. It helped break me out of analysis paralysis and move forward with implementing ideas I had been on the fence about.