Index

Note: Tables and boxes are indicated by t, and b following the page number

Note: For the benefit of digital users, indexed terms that span two pages (e.g., 52-53) may, on occasion, appear on only one of those pages.

1/N heuristic, defined, 301

A Beautiful Question (Wilczek), 245b

Adam, Robot Scientist

creating experiments, 388-89 formulating hypotheses, 389-90 thinking process of, 390-92

ADI (Argument Driven Inquiry), 334

Advice for a Young Investigator (Ramon y Cajal), 3

affirming the consequent, 24-25

Ahissar, E., 192n31

ahistorical science, 92

AI.

Alexander, S., 192n22

Alger, Bradley E., 276n22

algorithm dynamics, 377

Alvarez Hypothesis, 92

Amazon, Inc., Big Data Mindset and, 375

American Society for Cell Biology

(ASCB), 162-63

analytic reproducibility

defined, 162-63

reasons for, 163

in Reproducibility Project: Psychology, 170 Anderson, Chris, 374

Anderson, P. W., 105

applied science

Indigenous science and, 106-7

overview, 94

philosophy of action and, 69-70

Araque, Alfonso, 60n54

Argue, K.J., 372n12

Argument Driven Inquiry (ADI), 334 aristocratic induction

under-determinism and, 24-25

logical fallacy in, 25

Aronson, Elliott, 319

artificial intelligence (AI), 384-93

neural networks, 385-88

deep learning, 386 explainable AI (XAI), 387-88 overview, 384-85 Robot Scientists, 388-92

creating experiments, 388-89 formulating hypotheses, 389-90 thinking process of, 390-92

The Art of Insight in Science and Engineering (Mahajan), 370

ASCB (American Society for Cell

Biology), 162-63 Aschwanden, C., 192n22 The Astonishing Hypothesis (Crick), 109n6 atlases, standards of objectivity and, 8-9 automatic thinking, 278-95 biases, 288-90, 298-306

confirmation bias, 318-22 ecological rationality, 316-18 framing effects, 322-23 heuristics, 299-306 overview, 299-300 publication bias, 322-23 generating hypotheses, 279-88 from consciousness, 283-84 counterfactual thinking, 285-87 sensory illusions, 287-88 split brain studies, 281-83 from unconscious, 285 inductive reasoning, 292-94 overview, 278-79 rationality, 306-7

framing effects, 311-13 hypotheses and, 314-23 Linda problem, 307-9 overview, 306-7 probability versus frequency, 309-10 scientists and, 313-14

Wason Selection Task, 310-11 scientific fads and, 290-92

Ayer, A.J.

Babbage, Charles, 193n34

Bacon, Francis

enumerative induction, 17-18

Glass and, 256

“idols of the mind,” 280

view on hypotheses, 244b

Badcock, C., 325n23

Baker, M., 191n17

Bar, Moshe, 217n20

basic science distinguished from applied science, 94

Baxter, Louise M., 348n10

Bayes, Thomas

general discussion of, 143

Reproducibility Crisis and, 179-81

Bayes factor, 151

Bayesian networks, 381-84

Bayesian statistics, 143-58 frequentists and, 156-58 general discussion of, 120 objectives, 151 overfitting, 153 overview, 143-44 probability, 144-51 statistical hypotheses and, 153-56 deduction, 155 explanation, 156 falsification, 156 induction, 154-55

Structural Causal Model and, 381-84

Bayes' Theorem (Bayes' Rule)

overview, 145-48

rewriting, 148-51

Beard, Daniel, 105-6

The Beginnings of Infinity (Deutsch), 274 Begley, C.G., 190n6

behavioral economics, improving scientific thinking with, 362-67 advancing by trial and error, 363-64 generating multiple hypotheses, 366-67 sunk cost fallacy

ignoring, 364-65 opportunity cost and, 365-66 taking chances, 363

Belsky, W.C., 195n60

Benjamin, Daniel J., 140n19 biases, 298-306

cognitive bias, 299

confirmation bias, 318-22

hypotheses and, 319-22

loss aversion, 319

defined, 299-300

ecological rationality and, 316-18

Google Flu Trends (GFT), 378-79 heuristics, 299-304

bad cognitive bias, 303-4 bias-variance dilemma, 301-3,368-69 Expected Utility Theory, 304-6 fast and frugal program, 300-1 overview, 299-300

in machine learning context, 378 overview, 288-90

publication bias

framing effects, 322-23 labeling experiment outcomes and, 212 loss aversion, 322-23

rejecting hypotheses and, 248 in Reproducibility Project:

Psychology, 180-81

scientific fraud, 290

social biases, 299

survey of effect of hypotheses, 224-26 bias-variance dilemma

Google Flu Trends (GFT) and, 378 identifying hypotheses, 368-69 overview, 301-3

Bickle, John, 336-41

Big Data, 103-6

artificial intelligence (AI), 384-93 neural networks, 385-88 overview, 384-85

Robot Scientist, 388-92 general discussion of, 87-104 hypothesis-based Big Science and, 104 hypothesis-based Small

Science and, 104-5

overview, 96-98

survey of scientists using, 222-24 systems biology, 105-6

Big Data hubris, 376-77

Big Data, Little Data, No Data

(Borgman), 96-97

Big Data Mindset, 374-84

defined, 374-75

Google Flu Trends (GFT), 375-81 building in good bias, 378-79 overfitting and, 377-78 overview, 375-76 problems with, 376-77 purpose of, 379-81

overview, 374

Structural Causal Model, 381-84

Big Science

Discovery Science and, 99-100

Human Genome Project as, 99-100 overview, 96-98 biological science, 220 black box thinking and scientific hypotheses, 345-46

Black Box 'Thinking (Syed), 345-46 The Black Swan (Taleb), 262 blueprints, hypotheses as, 211-12 Bond, Michael, 326n40 Borgman, Christine L., 97/ Boyer, Pascal, 217n21

Brainder, 217n11 Branch, Glenn, 110n21 Brandstatter, E., 326n27 Brock, L.G., 276n19

Bursalou, Lawrence W., 295n6

Butler, D., 394n9

Button, Katherine, 166-67 Byrne, R.M.J., 296n23

Can Theories Be Refuted? (Harding), 84n28 cannabis plants

Indigenous science and, 107 productive ignorance and, 239 Casarett, David, 113n58 Case, Nicky, 398n2 Castelvecchi, D., 276n27 Cat, Jordi, 27n10 cause, as part of hypotheses, 34 Centaur Chess, 398 Central Limit Theorem (CLT), 122 CER (Claims-Evidence-Reasoning), 334 Chalmers, I., 192n25 Chick, C.F., 325n22 chi-square variable, 203 Cipriano, A., 111n45 Claims-Evidence-Reasoning (CER), 334 Clark, Ronald W., 30n59 Cleland, Carol, 61 climate change hypothesis, 339-41 clinical science, 94

The Clockwork Universe (Dolnick), 59n46 CLT (Central Limit Theorem), 122 Coe, Robert, 140n17 cognitive advantages of scientific

hypothesis, 206-16 aiding scientific thinking, 210-15 as blueprints, 211-12 communication, 214-15 conscious interpretative

interaction, 212-13

define success and failure in research, 212 memory and narrative structure, 213-14 self-organization, 213

multiple hypotheses, 209-10

objectifying problems, 207-9

overview, 206-7

cognitive bias, 299

cognitive dissonance, defined, 319

cognitive ease, 358-62

curse of knowledge, 358-59

defined, 358

doing premortem, 361

good ideas having considerable reach, 362

outside view of own thinking, 359 cognitive illusions, 303 Cohen, J., 141n24

Cohen's d index, 135-36

cold fusion, 291

collective empiricism, 8-9

Collins, Francis, 161

Comey, James, 297n32 communication

advantages of scientific

hypothesis, 214-15

survey of effect of hypotheses, 224-26 computational biology.

defined, 162-63

testing, 164

conditional probability, applying in Bayes'

Theorem, 145-47

Conditional Reasoning (Nickerson), 296n21 confidence, in theories, 71-72 confidence intervals, 136-38

for effect size, 137-38

overview, 135

confirmation bias, 318-22

defined, 318

hypotheses and, 319-22

loss aversion, 319

confirmatory studies, 95-96

confirming, defined, 21

Conjecture and Criticism, 264-74

critique of, 273-74

good explanations, 265-66

hypothesis testing and, 267-68

overview, 264-74

predictions and, 268

rejecting empiricism, 266-67

rejecting induction, 266-67

scientific progress and, 268-69 theoretical quantum mechanics, 269-73

Conjectures and Refutations. See Critical Rationalism

Consciousness Explained (Dennett), 278 consciousness, generating hypotheses from, 283-84

Consciousness (Koch), 295n11

consensus, scientific, 15

constraint, 52

container model of unity, 88 contents of science, defined, 66-68

Conway, Erik, 83n14 correlations, spurious, 377 corroborated hypotheses in Critical Rationalism, 44-45 objections to, 70-72 rational argument for, 78

Cosmic Microwave Background radiation, 94

Cosmides, Leda, 89 counterfactual thinking, 285-87

Crick, Francis, 109n6

Critical Rationalism, 39-40, 61-82 corroborated hypotheses, 44-45 elimination of induction, 40-41 falsification, 41-42 objections to

to corroborated hypotheses, 70-72 to elimination of induction, 68-69 to falsification, 61-75 holism and, 72-74 negative data, 74-75 philosophy of action, 69-70 rational argument, 76-82

overview, 39-40

revising versus rejecting hypotheses, 42-44 swans example, 39-40, 80-82 “tested-and-not-falsified” hypotheses, 44

Critical Rationalism (Miller), 59n34

Critical Thinking, Science, and Pseudoscience (Lack & Rousseau), 324n1

The Critical Thinking Toolkit (Foresman, Fost & Watson), 324n1

critique

of Conjecture and Criticism, 273-74

of Curiosity-Driven Science, 242-47 of QMB, 258-59

crucial experiment, 59n39

Curiosity-Driven Science, 238-54 as being fishing expedition, 253-54 critique of, 242-47 curiosity defined, 249

example of, 249-51

failure

hypothesis and, 251-53 importance of, 241

hypothesis and, 240-41

hypothesis testing and

differences between, 247-48 similarities between, 247-48 ignorance, 238-40

Scientific Method and, 241 similarities between QMB and, 262-64 curse of knowledge, defined, 358-59

Daitch, Vicki, 297n31

Darwins Dangerous Idea, reductionism, 11 Daston, Lorraine, 8

Davis, Kimiberly J., 348n8 Dawes, Robyn M., 349n23 decision-making.

deductive reasoning

Bayesian statistics, 155 philosophy and, 15-17

Strong Inference and, 46 “Deep Blue” IBM computer, 384 deep implicit hypotheses

Duhem-Quine thesis and, 72 overview, 54-55 deep learning, 386 Dehghani, Morteza, 104-5 demarcation problem, 87-88 Dennett, Daniel, 11, 278 determinism, reductionism and, 11 Deutsch, David. See also Conjecture and Criticism

developing hypotheses with constraints, 52 generality, 47

overview, 264-65 diagramming scientific hypotheses, 353-57

Diebold, Francis X., 112n46 Dingledine, Raymond, 314 directional tests, statistical hypotheses and, 116-17

direct reproducibility

defined, 162-63

testing for in Reproducibility Project:

Psychology, 169

validating tests with, 163 direct testing, 37-38

Discovery Science

Big Science and, 99-100

Human Microbiome Project, 101-2 hypothesis-based science versus, 99 implicit hypothesis and, 102-3 overview, 99

Small Science and, 100

survey of scientists using, 222-24, 226-27 Dolnick, Edward, 59n46

Dolphin, A.C., 195n66 double-blinded studies, 95 double-slit experiment, 271-72

Douven, I., 395n28

Duhem-Quine Thesis, 72-74

Eccles, John, 248

ecological rationality

biases and, 316-18

general discussion of, 307

Eddington, Arthur, 48

Edison, Thomas, 74

Edmonds, David, 57n19

education and scientific hypotheses, 329-47 black box thinking, 345-46

overview, 329-31

post-high school, 336-41 overview, 336 program of Giere, Bickle, and

Mauldin, 336-41

pre-college levels, 331-36 hypothesis defined in, 332-33

NGSS, 332

NSTA, 332

teaching plans, 334 using “fault words,” 334-36 professional scientists, 342-45

focus on scientific premise, 342-44 NIH helping educate, 344-45 overview, 342

suggestions for improving, 341-42 Edwards, Lillian, 395n22 effect size, 135-36

Cohen's d index and, 135-36

confidence intervals for, 137-38

general discussion of, 128

overview, 135

using in Reproducibility Project: Psychology, 169-70

Eidinow, John, 57n19

Einstein, Albert

non-obvious predictions, 48

positivism and, 4

The Elegant Universe (Greene), 109n7 elimination of induction

in Critical Rationalism, 40-41 objections to, 68-69 eliminative inference, 73 Ellis, George, 277n28 empirical content

of scientific hypotheses, 117

of statistical hypotheses, 117-18 empiricism

Conjecture and Criticism rejecting, 266-67 defined, 6

Deutsch and, 265

hypothesis definition and, 34 endowment effect, 322 enumerative induction, 17-18 Environmental Protection Agency (EPA), 181 Environment and Public Works (EPW), 181 errors of commission, defined, 129 errors of omission, defined, 129

Erzkurdia, I., 111n43

Etz, A, 179-81

European Union (EU), explainable AI and, 387-88 exclusion, Strong Inference and, 46 Expected Utility Theory

defined, 305b

overview, 304-6

rational behavior, 306-7 experimental hypotheses, implicit, 53-54 explainable AI (XAI), 387-88 explanations

Bayesian statistics, 156

for Conjecture and Criticism, 265-66 defined, 12

as defining property of hypotheses, 34 levels of organization, 13-14 explanatory inference

defined, 23-24 understanding science with, 73 exploratory studies, 95-96

Failure (Firestein), 241

failure, Curiosity-Driven Science and

hypothesis and, 251-53

importance of, 241

fallibilism, 40-41

defined, 7-8

Deutsch and, 273

falsification and, 62

philosophers accepting, 70 false negative rate, 131

false positive rate, 131

falsification

Bayesian statistics, 156 characteristics of good hypotheses, 52 in Critical Rationalism, 41-42 demarcation problem and, 87 in Indigenous science, 107-8 methodological unity founded on, 90 objections to, 61-75

lacking purpose, 63-64 levels of scientific analysis, 79-80 method and contents of science, 66-68 not decisive, 62-63

rejecting hypotheses, 64-65

probability and, 124-26

purpose of, 63

Robot Scientists and, 389

Fancourt, D., 113n59 fast-and-frugal program finding hypotheses, 368-70 overview, 300-1 role of emotion, 312-13

Fatt, Paul, 50

fault words

avoiding, 334-35

necessity of using, 335-36

rejection and, 335

Feyerabend, Paul, 5-6

Feynman, Richard

formulating hypotheses, importance of, 213 levels of organization, 13-14 viewpoint on philosophy, 4

Firestein, Stuart.

distinctions between hypotheses and models, 259

overview, 238

taking chances, 363

Fisher, Ronald A.

contributions to NHST program, 133

Method for Combining Probabilities advantages of, 205-6 example, 203-4 overview, 202-3

using significance level to reassess

PPV, 204-5

Neyman-Pearson versus, 132-33

null hypothesis, 126-27

overview, 126-27

significance test, 127-28

Fleischman, Martin, scientific irreproducibility, 297n36 folk psychology, 278

Fooled by Randomness (Taleb), 194n54, 324n9

Foresman, Galen A., 348n22

Fost, Peter S., 348n22

fraud, scientific, 290 framing effects

overview, 311-13 publication bias, 322-23 free will, 296n15 frequency, probability versus, 309-10 frequentist statistics, 126-33

Bayesian statistics and, 156-58

Fisher

Neyman-Pearson versus, 132-33 overview, 126-28

general discussion of, 120

Neyman-Pearson, 128-33

Fisher versus, 132-33

overview, 128-29

statistical errors, 129, 131-32 statistical power, 129-32 probability and, 121-22

Galison, Peter, 8 gambler's fallacy, 121 Gaussian distribution, 121 gaze heuristic, 300-1

Gazzaniga, Michael, 295n7

Geertz, Clifford, 110n23

gene counting, implicit hypothesis and, 102-3

Gelman, Andrew, 155,

General Theory of Relativity, 48

Genius (Gleick), 362

Gestalt psychologists, 280-81

GFT. See Google Flu Trends

Giere, Ronald, 336-41

Gigerenzer, Gerd

bias-variance dilemma, 301-3 distinguishing chance, 120 ecological rationality, 300 fast-and-frugal program, 300-1 gaze heuristic, 300-1

identifying hypotheses, 368-70

Gilbert, Daniel, 170-71

Gill, S.R., 111n37

Ginsburg, Jeremy, 393n6

Glass, David, 254-56

Gleick, James, 362

global warming, falsified hypothesis and, 65

Godfrey-Smith, Peter, 5

observations and theories, 24

Popper and, 62

Goldman, Steven L, 89

Good, Irving J., 158

Good 'Thinking (Good), 158

Google Flu Trends (GFT), 375-81

building in good bias, 378-79

as implicit hypothesis or prediction, 380b overfitting and, 377-78

overview, 375-76

problems with, 376-77 purpose of, 379-81

Gopnik, A., 28n36

Gorski, David, 43-44

grant applications, survey of scientific hypothesis in, 228-29

Greene, Brian, 109n7

Grice, H.P., 308, 325n20

Gross, Paul R., 109n16

Grubaum, Adolph, 84n27

Hacohen, Malachi, 57n18

Haggard, Patrick, 296n15

hard science, 91-92

Harding, Sandra, 9-10, 84n28

Hastie, Reid, 349n23

Hawking, Stephen W., 28n29

HBL (Hypothesis-Based Learning), 333 Helmholtz, Hermann, 280-81

Henderson, Leah, 29n40

Hertwig, Ralph, 325n21 heuristics, 299-304

1/N heuristic, 301

biases and

bad cognitive bias, 303-4 bias-variance dilemma, 301-3, 368-69 Expected Utility Theory, 304-6 fast and frugal program, 300-1

gaze heuristic, 300-1

overview, 299-300

Polya and, 368

Higgs boson, Big Science/Big Data testing, 104

Higgs Discovery (Randall), 141n21

Hines, W.C., 194n51

historical science, 92

Hoddeson, Lillian, 282 holism, objections to Critical

Rationalism, 72-74

How to Solve It (Polya), 367-68

Hubel, David, 37

Human Genome Project, 99-100 Human Microbiome Project, 101-2 Hume, David

analysis of mind, 280

validity of induction, 18, 25-26 Hypothesis-Based Learning (HBL), 333 hypothesis-based science

advantages and disadvantages of, 224-26 Big Science/Big Data testing, 104 Conjecture and Criticism and, 267-68 Curiosity-Driven Science and

differences between, 247-48 similarities between, 247-48

Discovery Science versus, 99 in grant applications, 228-29 influence of, 226-27 in journal articles, 228-29 in neuroscience literature, 229-31 opinions about, survey of, 220-29 overview, 98

Small Science/Big Data testing, 104-5 survey of scientists using, 222-24, 226-27

hypothesis-testing studies, 95

IA (intelligence augmentation), 398

IBM computers

“Deep Blue,” 384

“Project Debater,” 384

“Watson,” 384

Idols of the Cave, defined, 280 “idols of the mind,” defined, 280

Idols of the Tribe, defined, 280

ignorance, Curiosity-Driven Science and, 238-40

Ignorance (Firestein), 240

The Illusion of Free Will (Wegner), 295n12 illusions

cognitive, 303 sensory, 287-88 thermal, 287-88 visual, 287

implicit hypotheses, 53-55

Curiosity-Driven Science and, 253 deep, 54-55, 72 difficulty of finding, 352

Discovery Science and, 102-3 experimental, 53-54 gene counting and, 102-3

Google Flu Trends (GFT) as, 380b implicit hypotheses (cont.) Linda problem and, 209 Ramon y Cajal, Santiago, 53 theory laden observations and, 53 improving education about scientific hypotheses, 341-42 Indigenous science, 106-8 applied science and, 106-7 cannabis drug, 107 International Panel on Climate

Change, 106-7 overview, 106 Polynesian mariners, 106-7 using modern science and, 107 indirect testing, 37-38 inductive inference

Bayesians using, 154 defined, 23-24 inductive power defined, 261-62 overview, 255-56 inductive reasoning, 17-26 aristocratic induction, 24-25 under-determinism and, 24-25 logical fallacy in, 24, 25 automatic thinking, 292-94 Bayesian statistics and, 154-55 Bertrand, Russell, 20

Conjecture and Criticism and, 266-67 Conjecture and Criticism rejecting, 266-67 Critical Rationalism and, 39-41

David Hume, validity of induction, 18, 25-26 elimination of, 40-41

enumerative induction, 17-18 exploding computers and, 20-21 finding hypotheses with, 367 finding own hypotheses with, 367 inference versus, 23-24 other forms of, 21-22 as philosophical problem, 25-26 plebian induction, 24 Principle of Induction, 19-20 probability and, 22 probable truth, 23 Problem of Induction

overview, 18-19 proposed solutions to, 19-21 UN assumption and, 18

Salmon, Wesley, 77

inference, inductive reasoning versus, 23-24 informative priors, 147 inhibitory postsynaptic currents

(IPSCs), 250-51

instrumentalism, as substitute for good explanation, 268

integrated causal model of unity, 89-90 intelligence augmentation (IA), 398 interference pattern, multiverse theory and, 271-72

The Invention of Science (Wooton), 57n22 Ioannidis, John, 166-67

Ionian Enchantment, 86

IPSCs (inhibitory postsynaptic currents), 250-51

irreducible error, bias-variance dilemma and, 301-2

irreproducibility

arsenic-using bacteria, 291 cold-fusion, 291

importance for science, 168b

NIH and, 161

journal articles, survey of scientific hypothesis in, 228-29

justificationism, defined, 58n23

Kahneman, Daniel

confirmation bias, 321

heuristics, 300, 303-4 inside perspective, 359 outside point of view, 352 premortem, doing, 361

Kandel, Eric, 280-81

Kaplan, David, 158

Karl Popper (Hacohen), 57n18

Kashmerick, Martin, 105-6

Kasparov, Gary, 398

Katona, I., 372n11

Katz, Bernard, 50

Keeley, Page, 333

Kekule, August, 285

Keller, Asaf, 168b

Kimmelman, J., 110n27

King, R.D., 393n3

Koch, Christof, 295n11

Konicek-Moran, Richard, 348n14

Kosinski, Michael, 394n19

Kuangnov, Cliff, 395n21

Kuhn, Thomas, 5-6

changes in scientific attitudes, 65 scientific revolutions, 246

Kullmann, D.

Lacal, Irene, 83n19

Lack, Caleb W., 324n1

Lakatos, Imre, 5-6

Lakens, Daniel, 140n20

Laland, K., 83n20

Lamdin, C., 140n16

Landis, Story C., 84n30 Laney, Doug, 103 Laplace, Pierre-Simon, 143 Large Hadron Collider project, 249 Larmarckism, 65

Laudan, Larry, 5-6

belief that science leads philosophy, 4-5 view on enumerative induction, 17-18 law, scientific, similar to hypothesis, 56n3 Lazer, D., 394n7

Leek, J., 192n27 levels of organization explanation, 13-14 as “limiting cases” of general theories, 84-85n35 of nature, 11-12, 71 uncertainty, 13-14

levels of science, objections to falsification, 79-80

Levitt, Norman, 109n16

Libet, Benjamin, 283-84 Linda problem, 207-9, 307-9 Linus, Francis, 193n34 Little Data, 96-98

Loewi, Otto, 285

logical fallacy, in aristocratic induction, 24, 25 The Logic of Scientific Discovery (Popper), 62 long-term potentiation (LTP), 210 Lorsch, Jon, 162-63 loss aversion

confirmation bias, 319

hypotheses and, 317-18 publication bias, 322-23 sunk cost fallacy and, 364

LTP (long-term potentiation), 210

Mach, Ernst, 4

machine learning. See artificial intelligence Magee, Bryan, Popper's philosophy of action and, 53, 93-94

Maguire, Eleanor A., 296n22

Mahajan, Sanjoy, 370 Mailer, Norman, 218n28 malaria, using Indigenous and modern science to cure, 107

marginal sciences, 338-39

Marsicano, Giovanni, 355

Masri, R., 192n32

material challenges, associated with reproducibility, 165

mathematical models, systems biology

and, 105-6

matters of fact, 15-17

matters of reason, 15-17

Maucer, H.I., 194n46

Mauldin, Robert, 336-41

Mayer-Schonberger, Viktor, 393n4

Mayo, Deborah G. 58n27

Mayr, Ernst, 11-12

McComas, William F., 349n32 mechanical objectivity, 8-9 mechanism of action, defined, 12

Meehl, Paul, 141n26

memory, 213-14

Merchants of Doubt (Oreskes & Conway), 83n14 Mermin, N. David, 277n31

meta-cognitive approach, 278-79 meta-science

defined, 167-69

Reproducibility Project: Psychology, 169-79 all-or-none, 173-74

assumptions, 174

consequences, 174

failure of one result, 174-75

as meta-science subject, 172-73 multiple tests, 177-79

replicating experiments, 175-77 seeking truth, 179

method of science, objections to falsification, 66-68 methodological unity, 90-91

Michelson, Albert, 252-53

Michelson-Morely experiment, 252-53

Miller, David

Bayes' Theorem, reasoning with, 155 falsification, 67-68

restatement of Critical Rationalist

philosophy, 59n34 “tested-and-not-falsified” hypotheses, 44 Misbehaving: The Making of a Behavioral

Economist (Thaler), 326n43

models and hypotheses, 259

modern science

defined, 6-7

overview, 106-8

Motelow, J.E., 372n3

Mullally, Sinead L., 296n22

Muller-Lyer illusion, 287

multiple hypotheses

Chamberlin, T.C., 59n38 improving scientific thinking with, 366-67 Strong Inference, 46-47

multiverse theory

conjecture of, 269-73 defined, 264

narrative structure, hypotheses providing, 213-14

National Institutes of Health (NIH) hypotheses in grant applications, 257 lack of information about scientific hypothesis 344-45

overview, 342

rigor, 342-44 transparency, 342-44

National Science Foundation (NSF), 329-30

National Science Teachers Association (NSTA) education about scientific hypotheses, 332 general discussion of, 331

natural science, 93

Nature science journal

Google Flu Trends (GFT), 380b

survey on

reproducibility, 166

use of hypothesis, 229-31

negative data

defined, 74-75

objections to Critical Rationalism, 74-75 publication bias and, 322-23

neural networks in artificial intelligence, 385-88 deep learning, 386 explainable AI (XAI), 387-88

overview, 385-86

neuroscience

hypotheses and, 284 literature, scientific hypothesis in, 229-31 scientific fads and, 290-91

Never at Rest (Westfall), 59n44

The New Organon (Bacon), 244b

Newton, Isaac

Glass and, 256

inductive reasoning and, 18 view on hypotheses, 244-45b

Next Generation Science Standards (NGSS) education about scientific hypotheses, 332 general discussion of, 331 hypothesis defined in, 332-33 science education standards, 331-32

Neyman, Jerzy, 126. See also Neyman-Pearson program

Neyman-Pearson program, 128-33

Bayesian perspective and, 158

Fisher versus, 132-33

overview, 128-29

statistical errors, 129, 131-32

statistical power, 129-32

NGSS. See Next Generation Science Standards

NHST. See null hypothesis significance testing program

Nickerson, Raymond S., 296n21

Nicoll, Roger, 210

NIH. See National Institutes of Health

nil hypothesis, defined, 126-27 noise

bias-variance dilemma and, 301-2

in statistics, 119-20 non-hypothesis-based science

Discovery Science, 99

Big Science and, 99-100

Human Microbiome Project, 101-2 hypothesis-based science versus, 99 implicit hypothesis and, 102-3

Small Science and, 100

overview, 98 non-informative priors, defined, 148 Nonsense on Stilts (Pigliucci), 61 Nord, C.L., 192n28 normal distribution, 121 Nosek, Brian, 169 nowcasting, 375-76 NSF (National Science

Foundation), 329-30

NSTA. See National Science

Teachers Association

Nudge (Thaler & Sunstein), 326n33

null hypothesis

flaws in, 133-34

overview, 126-28

null hypothesis significance testing (NHST) program

Bayes' Theorem versus, 155

Fisher, Ronald A.

Neyman-Pearson versus, 132-33 overview, 126-28

as major statistical hypothesis testing mode, 158

Neyman-Pearson program, 128-33

Fisher versus, 132-33

overview, 128-29

statistical errors, 129, 131-32

statistical power, 129-32

overview, 133-35

obesity epidemic, 115-17 objective priors, 147

Objectivity, (Daston & Galison), 8 objectivity, 8-10

as core value in science, 7

hypotheses helping with, 207-9

in observations, 41

Objectivity and Diversity (Harding), 9-10 Occam's Razor, 48-50

Okasha, Samir; 27n2

1/N heuristic, defined, 301

one-tailed significance test, 116-17 open-ended questioning, survey, 222-24 Open Science Collaboration, 169 opioid addiction, Bayes' Theorem

and, 145-48

opportunity costs, sunk cost fallacy and, 365-66

Oreskes, Naomi, 276n26

Orians, Gordon F., 110n19

Orr, H. Allen, 108n2 overfitting

Google Flu Trends (GFT) and, 377-78 overview, 153

parsimony, rule of, 48-50 Pashler, H., 191n22

PDFs (probability density functions), defined, 144-45

Pearl, Judea, 381-84

Pearson, Egon, 126. See also Neyman-Pearson program

Penzias, Arno, 94

Perezgonzalez, J.D., 140n15

Petabyte Age, 374 philosophy, 3-27

attaining truth, reasons for, 14-15

Critical Rationalism, 39-40 deductive reasoning, 15-17 fallibilism, 7-8

inductive reasoning, 17-26 affirming consequent, 24-25 enumerative induction, 17-18 inference versus, 23-24 other forms of, 21-22 as philosophical problem, 25-26 probability and, 22 probable truth, 23

Problem of Induction, 18-21

influence of science on philosophy, 4-5 levels of organization explanation, uncertainty, and, 13-14 of nature, 11-12

matters of fact, 15-17

matters of reason, 15-17 modern science, 6-7 neuroscience and, 284 objectivity, 8-10 philosophy of science and, 5-6 reductionism, 10-11

science versus, 4-5

Scientific Method, 26

Philosophy and the Real World (Magee), 60n53

philosophy of nature, defined, 5 philosophy of science, 5-6 physicalism, reductionism and, 10-11 Picornavirus project, 100

Pigliuicci, Massimo, 61

Pinker, Steven, 358

Pinto, Y., 295n8

Planck, Max, 65

Platt, John, 45-47

mathematical models, 105-6

Popper versus, 76

Strong Inference, 45-47 deduction and exclusion, 46 multiple hypotheses, 46-47, 209 steps for, 46

plebian induction, 24

Poincare, Henri, 299

Polya, George, 367-68

Popper, Karl, 38-45

Critical Rationalism, 39-40 corroborated hypotheses, 44-45 elimination of induction, 40-41 falsifiability criterion, 41-42 revising versus rejecting hypotheses, 42-44

“tes ted-and-not-falsified” hypotheses, 44

Glass and, 257

influence on science, xxi

overview, 38-39

philosophy of action, 69-70

Platt versus, 76 probability and, 122-26 falsification, 124-26 hypothesis testing, 123-24 probable truth, 123

realism and, 77

Salmon's critique of, 77

Popper, Karl (cont.)

similarities between Deutsch and, 267-68 Pons, Sidney, scientific irreproducibility, 297n36 positive predictive validity (PPV)

advantages of reproducibility, 196-206 calculating, 198-99 multiple tests and higher, 199-206

Fisher's Method for Combining

Probabilities, 202-6 overview, 201 overview, 196-98 positivism, 4 posterior probability (posterior odds),

145-47

post-high school education and scientific

hypotheses, 336-41

overview, 336

program of Giere, Bickle, and

Mauldin, 336-41

power. See statistical power PPV. See positive predictive validity pre-college levels, education and scientific

hypotheses, 331-36

hypothesis defined in, 332-33

NGSS, 332

NSTA, 332

teaching plans, 334 using “fault words,” 334-36 avoiding, 334-35 necessity of using, 335-36 rejection and, 335 predictions

Conjecture and Criticism and, 268 hypotheses versus, 35-37 non-obvious, 48

obvious, 48

Predictions in the Brain (Bar), 217n20 premortems, scientific thinking and, 361 pre-registration

as confirmatory studies, 95 reproducibility and, 183-89 advantages of, 186 disadvantages of, 186-89

Principia Mathematica

(Newton), 244b

Principle of Induction, 19-20

Prinz, F., 190n7

priority heuristic, 312 prior probability

applying in Bayes' Theorem, 145-47 calculating PPV with, 197-98

priors

defined, 144-45

types of, 147-48

probability, 121-22

Bayesian, 144-51

frequency versus, 309-10

frequentists and, 121-22

inductive reasoning and, 22

Popper and, 122-26 falsification, 124-26 hypothesis testing, 123-24 probable truth, 123

positive predictive validity (PPV) and advantages of reproducibility, 196-206

calculating, 198-99 multiple tests and higher, 199-206 overview, 196-98

positive predictive value and, 196-97 quantum mechanics and, 124 probability density functions (PDFs), defined, 144-45

probable truth

inductive reasoning, 23

Popper and, 123

Problem of Induction

overview, 18-19

proposed solutions to, 19-21

UN assumption and, 18 procedural challenges, associated with reproducibility, 166 progress, scientific, Conjecture and Criticism and, 268-69

“Project Debater” IBM computer, 384 Prospect Theory

Linda problem, 307-8

rationality and, 306-7

role of emotion, 312-13

publication bias

framing effects, 322-23

labeling experiment outcomes and, 212 loss aversion, 322-23

rejecting hypotheses and, 248

in Reproducibility Proj ect:

Psychology, 180-81

p-values

in biological sciences, 131

defined, 125

misinterpretations of, 134-35 positive predictive validity (PPV) and advantages of reproducibility, 196-206

calculating, 198-99 multiple tests and higher, 199-206 overview, 196-98

reporting exact, 127-28

using in Reproducibility Project:

Psychology, 169-70

qualitative challenges of reproducibility, 165-66 quantum mechanics, probability and, 124 Questioning and Model-Building

(QMB),254-64

critique of, 258-59

distinguishing between models and hypotheses, 259-61

inductive power, 261-62

overview, 254-56, 257

rejecting hypotheses, 256-57 similarities between Curiosity-Driven

Science and, 262-64

Quine, W. V. O, 4-5

Ramon y Cajal, Santiago

implicit hypotheses, 53

importance of philosophy in science, 3

view on hypotheses, 206-7 Randall, Lisa, 276n20 rational argument, objections to Critical

Rationalism, 76-82

hypotheses as basis for practical action, 78-79

levels of scientific analysis, 79-80 observation of black swans, 80-82 truth of corroborated hypothesis, 78 Rational Choice in an Uncertain World

(Hastie & Dawes), 325n16 rational economic theory, 305b rationality, 306-7

as core value in science, 7

framing effects

overview, 311-13 publication bias, 322-23

hypotheses and, 314-23 biases and ecological

rationality, 316-18 confirmation bias, 318-22 critical scientific thinking, 315-16 publication bias, 322-23

Linda problem, 307-9

overview, 306-7 probability versus frequency, 309-10 scientists and, 313-14

Wason Selection Task, 310-11 realism, 6

defined, 4

Karl Popper and, 77

reductionism, 10-11

defined, 10-11

greedy, 11

P. W. Anderson, 105

unity of science and, 88

weak, 11

Rees, Martin, 276n20

Reiss, Julian, 28n16 rejecting hypotheses

objections to, 64-65

revising versus, 42-44 reproducibility, 161-89

advantages of, 196-206

affirming the consequent and, 183 analytic reproducibility

defined, 162-63 reasons for, 163 in Reproducibility Project:

Psychology, 170

challenges associated with, 164-66 material, 165 procedural, 166 qualitative, 165-66

conceptual reproducibility defined, 162-63 testing, 164

as core value in science, 7

defined, 162-64

direct reproducibility

defined, 162-63

testing for in Reproducibility Project: Psychology, 169

validating tests with, 163

in observations, 41

overview, 161-62 pre-registration, 183-89 advantages of, 186 disadvantages of, 186-89

Reproducibility Crisis, 166-69 Bayes and, 179-81 statistical power and, 166-67

Reproducibility Project:

Psychology, 169-79 all-or-none, 173-74 assumptions, 174

reproducibility (cont.)

consequences, 174 failure of one result, 174-75 as meta-science subject, 172-73 multiple tests, 177-79 replicating experiments, 175-77 seeking truth, 179

Secret Science Reform Act

of2015, 181-82

systematic reproducibility

defined, 162-63 hypotheses and, 163-64 varying value of, 182-83 Reproducibility Crisis, 166-69.See also reproducibility

Bayes and, 179-81

statistical power and, 166-67 survey of, 166, 220-21, 227 Reproducibility Project: Psychology (RPP), 169-79

all-or-none, 173-74

assumptions, 174 consequences, 174 failure of one result, 174-75 as meta-science subject, 172-73 multiple tests, 177-79 replicating experiments, 175-77 seeking truth, 179 research, defining success and failure in, 212 revising hypotheses, rejecting versus, 42-44 risks, defined, 120 Robot Scientists, 388-92

creating experiments, 388-89 formulating hypotheses, 389-90 thinking process of, 390-92 Rousseau, Jacques, 348n20 RPP. See Reproducibility Project: Psychology rule of parsimony, 48-50 Russell, Bertrand, 20

Salmon, Wesley, 77 sampling error, bias-variance dilemma and, 301-2

Samuels, R., 326n41 Sapolsky, Robert, 84n26 Schoen, Jan Henrik, scientific fraud, 297n33 Schooler, J.W., 191n13 science, 86-108, 219-32

Big Data, 103-6

general discussion of, 87-104 hypothesis-based Big

Science and, 104

hypothesis-based Small Science

and, 104-5

survey of scientists using, 222-24

systems biology, 105-6

defined, 7 hypothesis-based

advantages and disadvantages

of, 224-26

influence of, 226-27

in journal articles and grant applications, 228-29

in neuroscience literature, 229-31 overview, 98-103

survey of opinions about, 220-29

survey of scientists using, 222-24 Indigenous, 106-8

meta-science

defined, 167-69

Reproducibility Project:

Psychology, 169-79 modern, 6-7 non-hypothesis-based, Discovery Science, 99-103, 222-24

objectives, 93-98

basic science versus applied science, 94

Big Data/Little Data, 96-98

Big Science/Small Science, 96-98 confirmatory versus exploratory, 95-96 open-ended questioning, survey of, 222-24 philosophy versus, 4-5 subject matter, 91-93

hard versus soft, 91-92

historical versus ahistorical, 92

natural versus social, 93

training of scientists, survey of, 221-22 unity of, 86-91

container model, 88

demarcation problem, 87-88

integrated causal model, 89-90 methodological unity, 90-91

Science journal, survey on use of hypothesis, 229-31

Science Wars, 89

scientific fads, 290-92

scientific hypotheses, 31-56, 196-216 automatic thinking, generating from, 279-88 from consciousness, 283-84 counterfactual thinking, 285-87 sensory illusions, 287-88 split brain studies, 281-83

from unconscious, 285 characteristics of good hypotheses, 47-52

constraint, 52

falsifiability, 52

generality, 47-52

riskiness, 48

significance, 47-52

simplicity, 48-51

specificity, 51

cognitive advantages of, 206-16

aiding scientific thinking, 210-15 multiple hypotheses, 209-10 objectifying problems, 207-9 overview, 206-7

Curiosity-Driven Science

and, 240-41

defined, 32-35, 332-33

in education, 329-47

black box thinking, 345-46

overview, 329-31

post-high school, 336-41

pre-college levels, 331-36 professional scientists, 342-45 suggestions for improving, 341-42 finding own, 352-57, 367-71

developing insights, 370-71

diagramming, 353-57

enhancing problem-solving

ability, 367-68

fast and frugal hypotheses, 368-70

induction, 367

training suggestions for, 357

future of, 374-93

AI, 384-92

Big Data Mindset, 374-84 implicit, 53-55

deep implicit hypotheses, 54-55

experimental hypotheses, 53-54 opponents of, 237-75

Conjecture and Criticism, 264-74

Curiosity-Driven

Science, 238-54, 262-64

Questioning and

Model-Building, 254-64 passive voice and, 360-61 b Platt, John, 45-47 Popper, Karl, 38-45

Critical Rationalism, 39-45, 61-82 overview, 38-39

predictions versus, 35-37

QMB rejecting, 256-57

Scientific Method

Curiosity-Driven Science and, 241 defined, 32

education of at pre-college

level, 330-31

general discussion of, 26 survey regarding, 221-22, 225-26 statistical advantages of, 196-206

multiple testing, 201-6 reproducibility, 196-201

statistical hypothesis versus, 114-19 empirical content, 117-18 obesity epidemic example, 115-17 relationship to external world, 118-19 testing, 118

strengthened by experience, 78-79 survey of, 219-32

advantages and disadvantages

of, 224-26

Big Data, 222-24

Discovery Science, 222-24 influence of, 226-27 in journal articles and grant applications, 228-29 in neuroscience literature, 229-31 open-ended questioning, 222-24 opinions, 220-29 scientists training of, 221-22 using, 222-24

testing

direct, 37-38

indirect, 37-38

Scientific Method

Curiosity-Driven Science and, 241 defined, 32

education of at pre-college

level, 330-31

general discussion of, 26

survey regarding

importance of, 225-26 knowledge of, 221-22

scientific models, QMB and, 259-61 scientific premise, scientific hypotheses versus, 342-44

scientific progress

Conjecture and Criticism and, 268-69

Stuart Firestein's view of, 238-39

scientific revolutions, 246

scientific thinking, improving, 351-71 cognitive ease, avoiding, 358-62 curse of knowledge, 358-59 doing premortem, 361 good ideas having considerable reach, 362 outside view of own thinking, 359 cognitive lessons from behavioral economics, 362-67

advancing by trial and error, 363-64 generating multiple hypotheses, 366-67 sunk cost fallacy, 364-66 taking chances, 363

finding hypotheses, 352-57, 367-71 developing insights, 370-71 diagramming, 353-57 enhancing problem-solving

ability, 367-68

fast and frugal hypotheses, 368-70 induction, 367

training suggestions for, 357

overview, 351

scientists, education of scientific hypotheses for, 342-45

focus on scientific premise, 342-44

NIH helping educate, 344-45

overview, 342

Secret Science Reform Act of 2015, 181-82 seizures, 281-82

Sellars, Wilfrid, 5

Sense of Style (Pinker), 358

sensory illusions, 287-88

Seymour, B., 324n34

Shepard, Roger, 110n18 significance test, 127-28

Silk, J. 277n28

Silver, Nate, 158

Small Science

Discovery Science and, 100

overview, 96-98

Picornavirus project as, 100

social biases, 299

social sciences, 93

soft sciences, 91-92

Soldatova, L.N., 395n24

solipsism, falsifiability and, 87

Soltesz, I., 372n13

Soon, C.S., 282

Sorge, R.E., 194n50

split brain studies, 281-83

Sprenger, Jan, 28n16

spurious correlations from Big Data Mindset strategies, 377

statistical hypotheses, 114-38

Bayesian statistics and, 120, 153-56 deduction, 155

explanation, 156

falsification, 156 induction, 154-55

confidence intervals, 136-38

effect size, 135-36

Cohen's d index and, 135-36 confidence intervals for, 137-38 frequentists, 126-33

Fisher, 126-28

general discussion of, 120

Neyman-Pearson, 128-33

importance of statistics, 119-20

NHST program, 133-35

probability, 121-26

p-value, defined for science, 131 scientific versus, 114-19

empirical content, 117-18 obesity epidemic example, 115-17 relationship to external world, 118-19 testing, 118

Statistical Inference as Severe Testing (Mayo), 58n27

statistical power

Neyman-Pearson program, 129-32 positive predictive validity (PPV) and advantages of reproducibility, 196-206 calculating, 198-99 multiple tests and higher, 199-206 overview, 196-98

Reproducibility Crisis and, 166-67 statistics

advantages of with scientific hypothesis, 196-206

multiple testing, 201-6 reproducibility, 196-201

Bayesian, 120, 143-58

frequentists and, 156-58 objectives, 151 overfitting, 153 overview, 143-44 probability, 144-51 statistical hypotheses and, 153-56 frequentists, 126-33

Fisher, 126-28 general discussion of, 120 Neyman-Pearson, 128-33

importance of, 119-20

probability, 121-26

Steward, Oswald, 84n30

Stoned (Casarett), 113n58

Stroebe, W., 192n22

Strong Inference

eliminative inference and, 73

Platt, John

deduction and exclusion, 46

multiple hypotheses, 46-47

overview, 45-47

steps for, 46

Structural Causal Model, 381-84

The Structure of Scientific Revolutions (Kuhn), 5-6, 65, 352

subjective probability, 22

sunk cost fallacy

ignoring, 364-65 opportunity cost and, 365-66

Sunstein, C., 326n33

supercomputers, 397

swans as Critical Rationalism

example, 39-40 inductive power and, 262 objections to, 80-82 Syed, Matthew, 345-46 syllogism, 15-16 systematic reproducibility defined, 162-63 hypotheses and, 163-64 systems biology

mathematical models for, 105-6 overview, 105-6

Tagkopoulous, I., 297n40

take-the-best heuristic, 369

Taleb, Nicholas Nassim, 239

tautology, 16

Tavris, Carol, 319

Teachingfor Conceptual Understanding in Science (Konicek-Moran and Keeley), 333 teaching plans for education of scientific hypotheses, 334

testability, as core value in science, 7 testing

direct measurements, 37-38

indirect measurements, 37-38 scientific versus statistical hypothesis, 118 “tested-and-not-falsified” hypotheses in Critical Rationalism, 44 objections to, 68

Thaler, Richard, endowment effect, 322

Thales of Miletus, unity of science and, 86 theoretical quantum mechanics, Conjecture and Criticism and, 269-73

theory, similar to hypothesis, 56n3 theory-laden observations, 41-42, 53 Theory of Everything (ToE), 11

Theory of Everything (Hawking), 28n29 thermal illusions, 287-88

Thinking, Fast and Slow (Kahneman), 303 “tidy account,” scientific pluralism versus scientific monism, 88

Tinbergen, Niko, 11-12

Tooby, John, 89 training of scientists, survey, 221-22 transparency, Secret Science Reform Act of 2015, 182

True Genius (Hoddeson & Daitch), 297n31

Truth

attaining, reasons for, 14-15 fallibilism and, 7

hypothesis resulting from search for, 32

Tse, Peter Ulric, 296n15

Tsien, R.W., 195n62

Tu, Youyou, 106-7

Tulving, E., 295n5 Turnbaugh, P.J., 111n38

Tversky, Amos, 303-4 two-tailed significance test, 116-17

unconscious, generating hypotheses from, 285

under-determinism, aristocratic induction and, 24-25

Understanding Scientific Reasoning (Giere, Bickle, and Mauldin), 336-37

Uniformity of Nature (UN) assumption, 18-19 uninformative priors, 148 unity of science, 86-91 container model, 88 demarcation problem, 87-88 integrated causal model, 89-90 methodological unity, 90-91

utility, defined, 305b

Vanderkerckove, J, 179-80

Van Dorn, Kristy, 348n12 varying value of reproducibility, 182-83 verifiability, defined, 57n10 visual illusions, 287

Waller, Niels G., 141n27 Wang, Yilun, 394n19 Wason, Peter, 310, 319-21

Wason Selection Task, 310-11 “Watson” IBM computer, 384 Watson, Jamie Carlin, 372n4 wave-particle duality, 270-72 Wegner, Daniel M., 295n12 Weinberg, Steven, 4 Westfall Richard, 59n44 Whitehead, Alfred, 19

Wiesel, Torsten, 37

Wilczek, Frank, 245b

Wilson, E. O., 86

Wilson, Robert, 94

Wittgenstein’s Poker (Edmonds &

Eidinow), 57n19

Wolfe-Simon, F., 297n39

Wooton, David, 57n22

XAI (explainable

AI), 387-88