Facts have Challenge Ratings

http://slatestarcodex.com/2015/07/23/the-general-factor-of-correctness/

Alexander neglects the challenge rating of facts. Every issue has a CR which your general epistemic competence g is checked against. If g >> CR, you get the correct answer. (Some caveats.) The opinion of a low g-rating mind on a high-CR question is determined by tribe and chance. Obvs if there's a tribal signal to display they'll display it, otherwise it's random.

Scales are arbitrary but I pegged journalists at 1-2 which puts Moldbug's g rating around 90. Don't confuse it with a percentage, I'm aiming for 120. I would also peg Scott Alexander's but he lies too much, I can't be arsed to work out which of his beliefs are sincere and which he's just pretending to not know.

1. Five Thirty Eight is down the night before an election, so you search for some other good sites that interpret the polls. You find two. Both seem to be by amateurs, but both are well-designed and professional-looking and talk intelligently about things like sampling bias and such. The first site says the Blue Party will win by 5%; the second site says the Green Party will win by 5%. You look up the authors of the two sites, and find that the guy who wrote the first is a Young Earth Creationist. Do you have any opinion on who is going to win the election?

In general it depends on the relative CR ratings, which I don't know off hand.  Even if Creationism in fact has a low CR rating, this is not evidence the Green Party site knows what it's talking about; what Alexander thinks is 'talking intelligently about sampling bias' can and is usually learned by rote, and thus useless. They will have to show their work, in which case I can evaluate their work directly.

Creationism is red tribe, not blue, so it may be a declaration against interest, which gives a bonus to g estimate.

2. On the bus one day, you sit next to a strange man who mumbles about how Bigfoot caused 9-11 and the Ark of the Covenant is buried underneath EPCOT Center. You dismiss him and never see him again. A year later, you see on TV that new evidence confirms Bigfoot caused 9-11. Should you head to Florida and start digging?

Depends on the cost. It doesn't lower the probability but it doesn't raise it much either. If you were going to Florida with backhoes anyway...well, why not? If you're a regular person, no. People who mumble in public are showing ignorance of social rules, which has a very low CR. (Do I want fractional CR or do I want it logarithmic?) It is most likely caused by the fact that strange things remind you of other strange things, and thus when you think of solutions to strange thing A), strange thing B) is in the availability bias slot.

There's also the very likely case that the TV is mistaken; as above, journalists are lucky if they breach g level 2.

There is a very small probability that they're ignoring social rules because they have atypical values, and thus aren't being irrational. Never personally observed a real specimen, but they should theoretically exist.

3. Schmoeism and Anti-Schmoeism are two complicated and mutually exclusive economic theories that you don’t understand at all, but you know the economics profession is split about 50-50 between them. In 2005, a survey finds that 66% of Schmoeist economists and 33% of anti-Schmoeist economists believe in pre-Clovis settlement of the New World (p = 0.01). In 2015, new archaeological finds convincingly establish that such settlement existed. How strongly (if at all) do you now favor one theory over the other?

Given the past performance of economists-who-make-mainstream-news-so-that-we-know-there's-a-controversy, most likely they're both wrong anyway.

But neglecting that (obviously higher-than-SA's-g-rating-CR) fact, without understanding Schmoeism to a reasonable degree you won't know its CR. If Schmoeism's CR is lower than Clovis, then it is evidence for Schmoeism roughly to the same ratio the CR is lower. If CR is 0 then Schmoeism is all but guaranteed; if it's 1-2 points lower, then there's no evidence either way; and if it's 20 or more points the wrong side is 75% likely all shills.

If Shmoeism's CR is significantly higher than Clovis they don't even understand the question, and it might be unreasonable to assume they're arguing about it.

4. As with 3, but instead of merely being the pre-Clovis settlement of America, the survey asked about ten controversial questions in archaeology, anthropology, and historical scholarship, and the Schmoeists did significantly better than the anti-Schmoeists on 9 of them.

With more questions, the odds that one of them has high CR is better.
Of course real scholars go and look when they can, rather than trying to hack weird probabilities. If Clovis' CR is reasonably high - which is the only way a real scholar wouldn't already know the answer - then the odds that any of them have high-enough CR is still negligible.

--

The fact that facts have a challenge rating is probably a high CR fact. Estimate 105 +/- 25.

The only person I've ever noticed to buck the CR trend is Elizer Yudkowsky, who bucks it in both directions, so I'm not sure what to make of it.

The CR of religion is well above 120. Most likely 150-200 region, but could easily be 300 or higher.