自我介紹

陳紹慶
慈濟大學人類發展與心理學系專任副教授

開放科學中心大使
心理科學加速器創始成員；訓練委員會助理監督
scchen.com

Fork my slides on Github

我正在進行的研究主題

閱讀句子的心像模擬歷程

Orientation Stanfield & Zwaan (2001)

source: Bergen (2012)

Shape Zwaan, Stanfield, & Yaxley (2002)

source: Bergen (2012)

Color Connell(2005, 2007)

source: Bergen (2012)

潛在應用：人類語言的圖像式處理

Q1：你認為這樣的研究渉及捏造(fabrication)與篡改(falsification)嗎？

Diederik Stapel造假事件

Daryl Bem超感知覺研究爭議

Source: https://youtu.be/42QuXLucH3Q

Source: https://youtu.be/42QuXLucH3Q

Criticisms

JPSP rejected failed replication(Ritchie, Wiseman, & French, 2012)

HARKing (Hypothesizing After the Results are Known; Kerr, 1998)

End of Science?

How p-hacking destroy science?

Return power posing
Return pizza papers

Six ways to p-hacking

1. Stop collecting data once p <.05.
   
2. Analyze many measures, but report only those with p <.05.
   
3. Collect and analyze many conditions, but only report those with p < .05.
4. Use covariates to get p < .05.
5. Exclude participants to get p < .05.
6. Transform the data to get p < .05.

Nelson(2014)

QRP Checklist

Review process could harm research quality

推薦閱讀

(Chambers, 2018 London)

Return How to select my preregistration
Return RR: Take Home Message

Q1:操作有疑問的研究都是涉及捏造與篡改嗎？

Q2:經不起考驗的研究結果會傷害研究誠信嗎？

預先註冊(Preregistration):收集資料前，註冊的研究計畫沒有同儕評審

註冊報告(Registered Report):收集資料前，註冊的研究計畫有同儕評審

註冊再現研究(Registered Replication Report)

(Lindsay, Simons, & Lilienfeld, 2016)

預先註冊(Preregistration)

“The researcher creates as detailed a description of his or her plans for a study as possible and saves those plans in a time-stamped, uneditable archive. This record can be shared with reviewers, editors, and other researchers.” (Lindsay, Simons, & Lilienfeld, 2016)

My understanding: Preregistration and registered report in social science follow the framework of Randomized Controlled Trial but have a space for the exploratory. Transparent research process is the researchers have to manage after the research plan was frozen.

註冊你的專案

Open-Ended Registration

OSF-Standard Pre-Data Collection Registration

AsPredicated registration

Prereg Challenge

• Check the template.
  
• Encourage researchers conduct exploratory study in preregisration form.
  
• Submit your preregistered study to the eligible journal.
  
• Publish your study before Dec. 31, 2018. COS reward you US$ 1000.

How to select/create my preregistration form?

Return Evidence Pyramid

Pilot: Study without sample size justification; Secondary data set
EXP: Study with sample size justification

真正有用的預先註冊

No matter which form you register your plan, make everything you register in the form public.

不論使用什麼樣的預先註冊表格，研究完成後必須確保所有預定紀錄的內容永遠公開。

註冊報告(Registered Report)

AIMS Neuroscience submission pipeline

(Chambers, Feredoes, Muthukumaraswamy, & Etchells, 2014)

How to find and select RR journal

• COS RR service
• Latest summary table: Check list of 21 features; Pre-study peer review (stage 1 peer review) and Provisional pre-study acceptance (in-principle acceptance) are the minimal criteria.
• Turn your target journal RR: Templete letter to editor(s).

註冊再現研究(Registered Replication Report)

Daniel Simons on Pre-registration (from “Improving your statistical inferences”)

When publication bias joined:

APS RRR

• Psychological science should emphasize findings that are robust, replicable, and generalizable.

• Direct replications are necessary to estimate the true size of an effect.

• Well-designed replication studies should be published regardless of the size of the effect or statistical significance of the result.

Facial Feedback Effect

Take Home Message

Return Evidence Pyramid

Q3:我該不該改進研究操作？

Q4:我要從什麼地方開始改進？

降級版權力姿勢效應

Good theory, Bad evidence

• Failed replications (Ranehill, 2015)
• Publication bias (Carney, Cuddy, & Yap, 2015; Simmons & Simonsohn, 2017)
• Sloopy methods: p-hacking(Cherry picking) and HARKing (Bailey, LaFrance, & Dovidio, 2017; Cesario, Jonas, & Carney, 2017; Gronau et al., 2017; Jonas et al., 2017; Klaschinski, Schnabel, & Schröder-Abé, 2017; Latu, Duffy, Pardal, & Alger, 2017; Ronay, Tybur, van Huijstee, & Morssinkhof, 2016)
• Till present, there is a reproducible but slight feeling of power.

自砸招牌的飲食心理學大師

Wansink boasted how to publish paper by QRPs

(Original date: 2016/11/21)

Return How p-hacking destroy science

GRIM: A tool to check the spooky roundings of scaling data, less than 100 samples.

James Heathers	Nick Brown	Jordan Anaya	Tim van der Zee

All 4 Wansink-boasted papers have QRPs:

Questions in Smater Lunchrooms

• This paper had been retracted because they collected the data from kindergartens, not the elementary schools they claimed in their paper.
  
• Later we will know how the questioned data were detected.

請教長官：若類似情事發生於貴校，權責單位將如何處置？

本演講進行時，Brian Wansink還是康乃爾大學專任教授

Why “Prestigious” researchers don’t open?

小蝦米的逆襲

• 改變發表／補助機制的草根力量
• 資料品質鑑識工具
• 開放研究資料累積平台

Commitment to Research Transparency and Open Science

Peer Reviewers’ Openness Initiative

Statcheck

SPRITE

Curate Science

1. Transparency
2. Analytic reproducibility
3. Analytic robustness
4. Effect replicability

(LeBel, vanpaemel, McCarthy, Earp, & Elson, 2017)

Ego depletion (Chrome使用者要解除網頁安全性封鎖)

Mental simulation

如何分享你的資料與分析程序

• Share your analytic codes in your preregistered plan.
  
• Share your data and materials in the open repository.
  
• Request the supports of journal editors, funders, and institutions.

Q5:公開資料對我有何好處？

Q6:要如何公開資料才能提昇研究品質？

協作模式

• 臨時結盟
• 投案平台
• CERN model

Many Labs project

Klein et al (2014)

CRSP special issue(2017): power posing

Feeling of power: all partiipants	Feeling of power: unfamiliar only

Quentin et al. (2017)

The Collaborative Replication and Education Project (CREP)

Code of Conduct

“The Society for the Improvement of Psychological Science (SIPS) holds professional conferences and other events to enable its members to work to improve methods and practices in psychological science, build professional networks, and receive continuing education.”

SIPS meeting events

• Workshop
• Hachthon
• Unconference
• Lightning Talks
• Meetup
• Mentor/Mentee lunch

Psychological Science Accelerator

After you signed in PSA …

• I collect data for all, all collect data for me
• Review study submissions
• Serve on one of operational or advisory committees

• Provide feedback on the procedures, policies, and governance of the Accelerator

• Check PSA introductory paper

PSA sites map

Workflow of PSA projects

Moshont et al. (in press)

How PSA manage your project

• Project proposal has to be registered report.
• If project PI is single, the committees will assign one collobrator help compose the well-written proposal to be submitted.
• Lab PI decide if he/she will join this project at this time.

• Each participating lab takes the research ethic approval respectively.

• If I have more time, I’d like to share more about PSA and colloborative projects.

For your more information

提昇與維持研究品質是建立誠信的基礎

開放七訣(Back et al., 2017 @ LMU)

自由提問