longer.blue

RE: https://markrubin.substack.com/p/what-is-critical-metascience "So, in my view, “critical metascience” doesn't mean “metascientists who are critical” because metascientists are critical of themselves and others all the time. Instead, “critical metascience” refers to a special type of criticism that challenges metascience’s dominant ideas and assumptions and offers new, countervailing perspectives. Hence, the word “critical” refers to disruptive, paradigmatic criticism (e.g., “Questionable research practices [QRPs] are not necessarily problematic”) rather than more specific, conventional criticism that implicitly accepts the paradigm’s assumptions (e.g., “This measure of QRPs is not reliable”)." While you give some more specific ideas of what the subject matter of critical metascience is later on (essentially, the history and evidence of metascience, particularly of metascience's Lakatosian "hard core"), if you want to summarize it again like this in a future article, then I think you should clarify more: are you restricting your focus to metascientific assumptions which count as one or more of the following: fundamental, superficial, fashionable, empirical, conceptual, political, not shared by non-metascientific science...? Or do you make no such restrictions? Just as metascience inevitably involves assumptions in common with science, like some intuitions on approximately correct ways to apply mathematical models, critical metascience will share some assumptions with both metascience and science. "For its part, metascience champions scientific criticism. For example, it advocates adversarial collaborations, red teams, effective peer review, error detection, and self-correction, and it encourage scientists to “bend over backwards to show how [they]’re maybe wrong” (Feynman, 1974). Consequently, it should welcome critical metascience as a useful addition to its self-criticism toolkit. Of course, metascientists don’t always need to concede to critical metascience arguments, but they should at least engage with those arguments publicly, formally, and carefully (Longino, 1990, p. 78; O’Rourke-Friel, 2025; Rubin, 2023). "And, it takes two to tango! So, critical metascientists need to be prepared to reach out to metascientists in order to have productive exchanges rather than to simply criticize from afar (for a good example, see Pham & Oh’s [2021a, 2021b] discussion with Simmons et al. [2021a, 2021b] about preregistration). They should also be careful not to artificially homogenize metascience or criticize straw-person or outdated versions of metascience arguments (Field, 2022; Rubin, 2023; Syed, 2025). Finally, they should be reflexive, continuously questioning their own perspectives, biases, and positionality (Bashiri et al., 2025, p. 7)." An important qualification is that this can only work well in a proven high-trust environment. It's painfully easy to get a lot of garbage published and taken seriously, especially if the crucial bullshit is covered up in truths. And thanks to things like bullshit asymmetry and fashionable ideologies, anyone responding to those may get overwhelmed pretty harshly. This is relevant to the question of what the relationships between science, metascience and critical metascience should be if we want to do science well. That's part of why I want to see more simulations of scientific processes as an evolving collaborative and adversarial network. Is it reasonable to assume that scientists operate in a high-trust environment? Do they need one? Can we have comprehensively-predictive science, communicated widely and well, despite what I see as huge incentives to produce propaganda for plainly evil and exclusionary causes? The shitticane imposed by the US government is part of a long history of political impositions on science, which was never very independent from politics to begin with despite science's potential for relatively uncontroversial merit. I wrote https://longer.blue/posts/u3bpmrgcg7 to summarize this.