故意找錯練習Spot the Error
本頁把 askLLM/docs/LIMITATIONS.zh-TW.md 裡記錄的錯誤與示警，改寫成一組「找碴」 練習：你會先看到一段情境與一段 AI 回覆，回覆裡藏著一個錯誤，練習先自己找出 問題所在，再打開答案核對。這是練習 verify/workflow.qmd 裡 V1（路徑核對）、 V3（範圍核對）與 V4a（讀碼核對）的具體方式——目的是讓你親身發現幻覺， 而不是被動接受文件裡的結論。
This page turns the errors and warnings recorded in askLLM/docs/LIMITATIONS.zh-TW.md into a “spot the error” practice set: you will see a scenario and an AI reply with an error hidden in it, try to find the problem yourself, then open the answer to check. This is a hands-on way to practice V1 (path check), V3 (scope check), and V4a (code-read check) from verify/workflow.qmd. The point is to discover the hallucination yourself rather than take the documentation’s word for it.
題目來源說明：第 1–6 題逐字取材自 askLLM/docs/LIMITATIONS.zh-TW.md 表格中記錄的實測錯誤路徑，出處已在每題標明。第 7 題到第 10 題都是示意範例， 題目本身會清楚標示。第 7 題改寫自 LIMITATIONS〈綜合使用建議〉第 3 點，該處的 「8 個樣本、Saguaro／Palo Verde／Ironwood」是為說明錯誤型態而寫的虛構例子， LIMITATIONS 原文已明文標示它不是實測記錄。第 8–10 題是本頁自行設計，用來練習 V4a 的程式碼紅旗核對，依 askLLM/R/llm-adapter.R 明文禁止的指令清單改寫成 教學用例。這四題都不是任何 AI 的真實輸出。第 11 題取自本站自己的實測記錄 （data/ 底下的 .omv 與 data/replies/ 的回覆全文）。目前共 11 題，本頁會在 課堂驗證後補進新的實測案例持續擴充。
第 1–6 題與第 11 題的 AI 回覆逐字引用自實測記錄，未經任何文字修飾，因此其標點 與句式不遵循本站的散文規範（Pinker Rules 7–10）；第 7–10 題的示意回覆則依這套 規範改寫。
Where the questions come from: Questions 1–6 are taken verbatim from the real error paths recorded in the tables of askLLM/docs/LIMITATIONS.zh-TW.md; the source is noted on each question. Questions 7 through 10 are illustrative examples, and each is clearly labeled. Question 7 is adapted from point 3 of the “overall recommendations” in LIMITATIONS, where “8 samples across Saguaro / Palo Verde / Ironwood” is a made-up example written to show the failure mode. LIMITATIONS itself now says plainly that it is not a recorded test result. Questions 8–10 are designed for this page to practice the V4a code red-flag check, built from the instruction list that askLLM/R/llm-adapter.R explicitly forbids. None of these four is real output from any AI. Question 11 comes from this site’s own test records, the .omv files under data/ and the full replies in data/replies/. There are 11 questions here today, and the set will grow as classroom use turns up new cases.
The AI replies in questions 1–6 and question 11 are quoted verbatim from the real-test record, with no stylistic editing, so their punctuation and sentence patterns do not follow this site’s prose style guide (Pinker Rules 7–10); the illustrative replies in questions 7–10 have been rewritten to follow it.
第 2 題：相關性長在哪裡Question 2: where correlation lives
來源：askLLM/docs/LIMITATIONS.zh-TW.md 第 1 節實測記錄。
情境：使用者問「我想看兩個連續變項的相關係數，該用哪個分析？」
AI 回覆：「請用 探索 > 相關性。」
問題：這條路徑錯在哪裡？
Source: recorded observation in section 1 of askLLM/docs/LIMITATIONS.zh-TW.md.
Scenario: A user asks “I want the correlation between two continuous variables — which analysis?”
AI reply: “Use Exploration > Correlation.”
Question: What is wrong with this path?
答案：相關矩陣實際位於 Regression 選單之下，不在 Exploration。跟 第 1 題一樣，AI 正確辨識出「該做相關分析」，但把它掛在錯誤的頂層選單下。 對應查核步驟：V1 路徑核對。
Answer: the correlation matrix actually lives under the Regression menu, not Exploration. As in question 1, the AI correctly identified “this calls for a correlation analysis” but hung it under the wrong top-level menu. Verification step: V1, path check.
第 3 題：主成分分析躲在哪裡Question 3: where PCA hides
來源：askLLM/docs/LIMITATIONS.zh-TW.md 第 1 節實測記錄。
情境：使用者問「我想對這幾個連續變項做主成分分析。」
AI 回覆：「可以到 探索 > 主成分分析。」
問題：這條路徑錯在哪裡？
Source: recorded observation in section 1 of askLLM/docs/LIMITATIONS.zh-TW.md.
Scenario: A user asks “I want to run a principal component analysis on these continuous variables.”
AI reply: “Go to Exploration > Principal Component Analysis.”
Question: What is wrong with this path?
答案：PCA 實際位於 Factor 選單之下，不在 Exploration。同一個錯誤 模式再出現一次：分析名稱對，選單掛錯地方。對應查核步驟：V1 路徑核對。
Answer: PCA actually lives under the Factor menu, not Exploration. The same error pattern shows up again: the analysis name is right, the menu it’s hung under is wrong. Verification step: V1, path check.
第 5 題：編出來的巢狀層級Question 5: an invented nesting
來源：askLLM/docs/LIMITATIONS.zh-TW.md 第 1 節實測記錄。
情境：使用者用英文問 AI 如何做線性迴歸。
AI 回覆：「Use Regression > General Linear Model > Linear Regression.」
問題：這條路徑錯在哪裡？
Source: recorded observation in section 1 of askLLM/docs/LIMITATIONS.zh-TW.md.
Scenario: A user asks the AI, in English, how to run a linear regression.
AI reply: “Use Regression > General Linear Model > Linear Regression.”
Question: What is wrong with this path?
答案：LIMITATIONS 記錄這是「巢狀層級是編的」——Linear Regression 確實 在 Regression 選單之下，但中間多插了一層不存在的 General Linear Model 子選單。這種錯誤特別難抓，因為頭尾都對，只有中間一層是編的；逐條核對每一層 （不是只看第一層與最後一層對不對）才抓得到。對應查核步驟：V1 路徑核對。
Answer: LIMITATIONS records this as “a fabricated nesting level.” Linear Regression really is under Regression, but an extra, nonexistent General Linear Model sub-menu has been inserted in between. This kind of error is especially hard to catch because the first and last segments are both correct. Only the middle level is invented. You only catch it by checking every level of the path, not just whether the first and last segments look right. Verification step: V1, path check.
第 6 題：連翻譯都出錯Question 6: even the translation is wrong
來源：askLLM/docs/LIMITATIONS.zh-TW.md 第 1 節實測記錄。
情境：使用者用中文問 AI 如何做線性迴歸。
AI 回覆：「請到 迴歸 → 線性線性(Linear Regression)。」
問題：除了路徑本身，這則回覆還有什麼地方看起來不對勁？
Source: recorded observation in section 1 of askLLM/docs/LIMITATIONS.zh-TW.md.
Scenario: A user asks the AI, in Chinese, how to run a linear regression.
AI reply: “Go to 迴歸 → 線性線性(Linear Regression)” (the Chinese label literally reads “linear-linear”).
Question: Besides the path itself, what else looks off about this reply?
答案：LIMITATIONS 特別點出「連中文標籤都出現錯字」——線性線性 是重複 詞，不是任何 jamovi 介面上真實出現過的字串。這提醒我們：路徑核對不能只看 「大致講得通」，連標籤的逐字拼寫都要對得上你自己 jamovi 裡看到的文字，因為 自信的語氣與正確性無關。對應查核步驟：V1 路徑核對。
Answer: LIMITATIONS specifically flags that “even the Chinese label has a typo.” 線性線性 (“linear-linear”) is a duplicated word that never appears anywhere in jamovi’s actual interface. This is a reminder that the path check isn’t satisfied by something that “roughly makes sense.” The label’s exact wording has to match what you see in your own jamovi. Confident tone has nothing to do with correctness. Verification step: V1, path check.
第 7 題（示意範例）：沒附摘要時的憑空生成Question 7 (illustrative): fabrication with no summary attached
本題為示意範例，並非任何 AI 的真實輸出。LIMITATIONS 用這個虛構例子 說明「沒有資料時模型會編一份出來」這種錯誤型態。
來源：askLLM/docs/LIMITATIONS.zh-TW.md〈綜合使用建議〉第 3 點的示意範例。
情境：使用者在送出問題時沒有勾選「Attach data summary to prompt」， 資料集其實是 iris（150 筆鳶尾花觀測，三個品種：setosa、versicolor、 virginica），問 AI「這份資料有幾個樣本、有哪些組別？」
AI 回覆（示意）：「這份資料大約有 8 個樣本，分成 Saguaro、Palo Verde 與 Ironwood 三個物種。」
問題：這則回覆錯在哪裡？根本原因是什麼？
This question is an illustrative example, not real output from any AI. LIMITATIONS uses this made-up case to show the failure mode: given no data, the model makes some up.
Source: the illustrative example in the “overall recommendations” section of askLLM/docs/LIMITATIONS.zh-TW.md, point 3.
Scenario: A user submits a question without ticking “Attach data summary to prompt.” The dataset is actually iris (150 observations, three species: setosa, versicolor, virginica), and the user asks “how many samples are in this dataset, and what groups does it have?”
AI reply (illustrative): “This dataset has roughly 8 samples, split into three species: Saguaro, Palo Verde, and Ironwood.”
Question: What is wrong with this reply? What is the root cause?
答案：這是 LIMITATIONS 用來說明錯誤型態的示意範例。未勾選「Attach data summary」時，AI 完全看不到你的資料，卻仍然給出具體數字（8 個樣本）與具體 物種名稱（Saguaro、Palo Verde、Ironwood，這三者都不是 iris 的品種，是憑空 生成的仙人掌／植物名稱）。根本原因不是「AI 算錯了」，是它從頭到尾沒有拿到任何 資料資訊，卻用跟有資料時同樣肯定的語氣回答。對應查核步驟：V3 範圍核對 ——回覆聲稱知道它其實看不到的東西，就要整段打折扣，並回頭確認送出問題時是 否真的附上了資料摘要。
Answer: LIMITATIONS uses this made-up case to show the failure mode. With “Attach data summary” unticked, the AI has no visibility into your data at all, yet it still produces a specific number (8 samples) and specific group names. Saguaro, Palo Verde, and Ironwood are not iris species at all. They are fabricated plant names. The root cause is not “the AI miscalculated.” It never received any information about the data in the first place, yet answered with the same confident tone it would use if it had. Verification step: V3, scope check. Discount entirely any reply claiming to know something it cannot actually see, then go back and confirm whether the data summary was actually attached when you submitted the question.
第 8 題（示意範例）：程式碼裡的安裝指令Question 8 (illustrative): an install call in the code
本題為示意範例，並非任何 AI 的真實輸出，用來練習 V4a 讀碼核對。
情境：使用者用 R code tutor 問「幫我對 data 裡的 score 欄位做常態性 檢定。」
AI 回覆（示意）：
install.packages("moments")
library(moments)
agostino.test(data$score)問題：這段程式碼違反了哪一條紅旗規則？
This question is an illustrative example, not real output from any AI, used to practice the V4a code-read check.
Scenario: A user asks the R code tutor “run a normality test on the score column in data.”
AI reply (illustrative):
install.packages("moments")
library(moments)
agostino.test(data$score)Question: Which red-flag rule does this code violate?
答案：第一行 install.packages("moments") 直接違反 askLLM/R/llm-adapter.R 第 172–177 行明文禁止的指令清單。System prompt 明確要求「never suggest install.packages（）」。這行不能執行。你的 Rj 環境不保證有網路權限或寫入 權限去安裝新套件。askLLM 應該只引用 <rj_environment> 清單裡已經 裝好的套件。對應查核步驟：V4a 讀（見 checklist-rtutor.qmd 的紅旗 字串表）。
Answer: askLLM/R/llm-adapter.R lines 172–177 explicitly forbid this instruction. The first line, install.packages("moments"), violates it directly. The system prompt states plainly: never suggest install.packages(). This line cannot run safely. Your Rj environment offers no guarantee of network access or write access to install a new package. askLLM must cite only packages already installed, from the <rj_environment> list. Verification step: V4a, read (see the red-flag table in checklist-rtutor.qmd).
第 9 題（示意範例）：程式碼裡的檔案讀取Question 9 (illustrative): a file read in the code
本題為示意範例，並非任何 AI 的真實輸出，用來練習 V4a 讀碼核對。
情境：使用者用 R code tutor 問「幫我畫出兩個變項的散佈圖。」
AI 回覆（示意）：
mydata <- read.csv("C:/Users/me/Desktop/survey.csv")
plot(mydata$x, mydata$y)問題：這段程式碼違反了哪幾條紅旗規則？（不只一條）
Scenario: A user asks the R code tutor “plot a scatterplot of two variables.”
AI reply (illustrative):
mydata <- read.csv("C:/Users/me/Desktop/survey.csv")
plot(mydata$x, mydata$y)Question: Which red-flag rules does this code violate? (More than one.)
答案：這一行同時踩到兩條紅旗。第一條是 read.csv(...)：程式碼不得 建議讀取檔案，資料一律要用既有的 data 物件。第二條是絕對路徑： C:/Users/me/Desktop/... 這種路徑因人而異，在你的機器上幾乎必然不存在。 你的桌面上也許剛好有一份叫 survey.csv 的檔案，但這段程式碼依然完全沒有 用到 jamovi 已經幫你準備好的 data 物件，等於繞過了 askLLM 的設計邊界。 對應查核步驟：V4a 讀。
Answer: this single line trips two red flags. The first is read.csv(...): no file reads are allowed, and the data must always come from the existing data object. The second is the absolute path: C:/Users/me/Desktop/... is machine-specific and almost certainly does not exist on your computer. Your desktop might happen to hold a file called survey.csv. This code still never touches the data object jamovi already prepared for you, and it sidesteps askLLM’s design boundary. Verification step: V4a, read.
第 10 題（示意範例）：引用一個沒裝的套件Question 10 (illustrative): citing an uninstalled package
本題為示意範例，並非任何 AI 的真實輸出，用來練習 V4a 讀碼核對。
情境：使用者的 Rj 環境裡只裝了 base R（沒有額外安裝任何統計套件）， <rj_environment> 清單因此只列出 base R 套件。使用者問「幫我做一個穩健迴歸 （robust regression）。」
AI 回覆（示意）：
library(MASS)
fit <- rlm(y ~ x, data = data)
summary(fit)問題：這段程式碼有問題嗎？如果 MASS 沒有出現在 <rj_environment> 清單裡，你會怎麼處理？
Scenario: The user’s Rj environment has only base R installed (no extra statistical packages), so the <rj_environment> list contains only base R packages. The user asks “run a robust regression for me.”
AI reply (illustrative):
library(MASS)
fit <- rlm(y ~ x, data = data)
summary(fit)Question: Is there a problem with this code? If MASS does not appear in the <rj_environment> list, what should you do?
答案：這裡要看清楚：MASS 隨 base R 一起發行，在多數 R 安裝裡本來就 存在，所以「引用 MASS」本身不一定是錯的。關鍵是核對，不是背名單。 真正的檢查方法是：打開這次呼叫附給 AI 的 <rj_environment> 區塊，逐字確認 MASS 是否真的列在裡面。如果列在裡面，這段程式碼沒有違反紅旗規則；如果沒 列在裡面（代表你的 Rj 掃描結果不含它），這段程式碼一執行就會得到 there is no package called 'MASS'。這種情況下應該回頭修改問題（例如 「只用 base R 做穩健估計」），而不是自己手動安裝套件來配合 AI 的回覆。 對應查核步驟：V4a 讀 與 V4b 跑（V4b 的錯誤訊息正是拿來確認 V4a 判斷是否正確的依據）。
Answer: look carefully here. MASS ships with base R and exists in most R installations by default, so citing MASS is not automatically wrong. The point is to check, not to memorize a blocklist. The real check is: open the <rj_environment> block sent to the AI for this call, and confirm verbatim whether MASS is actually listed. If it is, this code does not violate a red flag. If it is not listed, meaning your Rj scan didn’t find it, running this code will immediately produce there is no package called 'MASS'. In that case, go back and rephrase the question, for example “use only base R for a robust estimate,” rather than manually installing a package just to make the AI’s reply work. Verification steps: V4a, read and V4b, run (the V4b error message is exactly what confirms whether the V4a judgment was correct).
第 11 題：同一個模型，兩種問法Question 11: same model, two ways of asking
本題取自
data/mixed-anova-setup_zhang2014_v1_fail_zhTW.omv的實測記錄， 回覆逐字引用，未經修飾。
情境：使用者請 R code tutor 幫忙把寬格式資料轉成長格式，並在提示詞裡要求 「只給程式骨架與 TODO 註解」。模型是 gemini-flash-latest。
AI 回覆（節錄）：
data_long <- data %>%
pivot_longer(
cols = c(# TODO: 填入要轉換的兩個時間點欄位名稱),
names_to = "# TODO: 指定時間點的新欄位名稱 (例如 'Time')",
values_to = "# TODO: 指定測量分數的新欄位名稱 (例如 'Interest')"
)問題：這段程式碼貼進 Rj 會發生什麼事？錯在模型，還是錯在問法？
This question comes from the recorded test in
data/mixed-anova-setup_zhang2014_v1_fail_zhTW.omv. The reply is quoted verbatim, with no editing.
Scenario: The user asked the R code tutor to reshape wide data into long format, and the prompt asked for “only a skeleton with TODO comments.” The model was gemini-flash-latest.
AI reply (excerpt):
data_long <- data %>%
pivot_longer(
cols = c(# TODO: fill in the two time-point column names),
names_to = "# TODO: name for the new time column",
values_to = "# TODO: name for the new score column"
)Question: What happens when this is pasted into Rj? Is the fault the model’s, or the question’s?
答案：貼進 Rj 會立刻得到 <text>:23:1: unexpected symbol，程式碼一行都跑 不了。原因是 R 的 # 會讓該行剩下的所有內容成為註解——包括 c( 需要的那個 右括號。運算式因此永遠收不了尾，解析器在後面第一個非空白符號處報錯。英文版的 同一次實測更嚴重，還出現 table(data_long$# TODO: ...)。
錯的是問法。 提示詞說「只給程式骨架與 TODO 註解」，等於邀請模型把註解填在 該放值的位置。同一個模型、同一份資料、同一個 persona、同一種提示詞語言，只把 那一句改成「待填處寫成開頭的變數指派、值放字串佔位符」之後，產出就變成：
target_cols <- c("TODO_填入T1欄位名稱", "TODO_填入T2欄位名稱")
time_col_name <- "Time"
data_long <- data %>%
pivot_longer(cols = all_of(target_cols), names_to = time_col_name, ...)待填處還在，但它們是字串，程式碼在你填值之前就能通過語法檢查。實測結果從 fail 變成 pass（見提示詞庫該條目的 tested_with）。
這一題真正要示範的不是那個 bug，而是查核方法本身：兩次執行只差一個因子， 所以差異可以歸因。這正是 workflow.qmd 的 cross-check 與第六課 「換模型或換 persona 重問一次」在教的事。如果那兩次同時換了模型又換了問法， 你就無法說出是哪一個造成差異。
對應查核步驟：V4a 讀 與 V4b 跑。這也是為什麼 V4b 不能省略——這段程式碼 「讀起來」很合理，錯誤只有執行才會現形。
Answer: pasting this into Rj immediately produces <text>:23:1: unexpected symbol, and not a line of it runs. R’s # turns everything remaining on that line into a comment, including the closing bracket that c( needs. The expression never terminates, and the parser reports the error at the first non-blank symbol after it. The English run of the same test was worse still, producing table(data_long$# TODO: ...).
The fault is the question’s. The prompt asked for “only a skeleton with TODO comments,” which invites the model to put comments where values belong. Same model, same data, same persona, same prompt language: change that one sentence to “put every blank as a variable assignment at the top, with a string placeholder as its value,” and the output becomes:
target_cols <- c("column name here", "column name here")
time_col_name <- "Time"
data_long <- data %>%
pivot_longer(cols = all_of(target_cols), names_to = time_col_name, ...)The blanks are still there, but they are strings, so the code parses before you fill anything in. The recorded result went from fail to pass (see the entry’s tested_with in the prompt library).
What this question really demonstrates is not the bug but the verification method. The two runs differed by exactly one factor, so the difference can be attributed. That is what the cross-check in workflow.qmd and Lesson 6’s “ask again with a different model or persona” are teaching. Had those two runs changed the model and the wording, you could not say which one caused the difference.
Verification steps: V4a, read and V4b, run. This is also why V4b cannot be skipped: the code reads perfectly sensibly, and the error appears only when it runs.