From www.astrology-and-science.com         Click here for home page or fast-find index

Effect sizes
How astrology compares with other approaches

Abstract -- An introduction to effect sizes in astrology and how they compare with those in other approaches such as graphology, palmistry, and psychological tests. Here effect sizes are expressed as a correlatiion, which ranges from +1 (perfect correlation as in feet vs metres) through 0 (no correlation as in two coin tosses) to -1 (perfect inverse correlation as in nightlength vs daylength). The results are also shown in terms of the equivalent hit rates, which for readers unfamiliar with correlations may be easier to understand. Astrology is said to fit reality so well that its effect size must be close to +1, equivalent to a hit rate close to 100%. But examples show how the fit is due to two non-astrological factors, namely (1) the ambiguity of astrological symbolism, so any chart can be made to fit any person after the event, and (2) hidden persuaders such as generalities-that-we-see-as-meaningful (the Barnum effect), so any person can see something that fits them in any reading. These factors make astrology seem to work even when the wrong chart is used. When these factors are controlled, for example by asking astrologers to match charts to owners whose identity is concealed, the observed effect sizes are not usefully different from zero. Accepted approaches such as personality and aptitude tests are better, usually much better. Especially telling is the negligible agreement between astrologers, because astrology seems pointless if astrologers cannot usefully agree on what a given birth chart indicates. Nevertheless astrology offers images of undeniable appeal that can feed the inner person in the same way that music, poetry and art do. There is more to astrology than having a useful effect size.

This article looks at effect sizes in astrology when expressed as a correlation, which indicates the extent to which A is related to B. Effect sizes range from +1 (perfect correlation as in feet vs metres) through 0 (no correlation as in two coin tosses) to -1 (perfect inverse correlation as in nightlength vs daylength). However, some readers may find it easier to see a correlation in terms of its corresponding hit rate, where x% hits means you are right x% of the time vs 50% expected by chance. Examples of the correlations observed in human affairs and their corresponding hit rates are shown below:

1.00 Perfect accuracy (100% hits)
0.95 Arm lengths left vs right (97.5% hits)
0.90 Tide height vs moon position (95%)
0.70 Adult height vs weight (85%)
0.60 Educational attainment of husbands and wives (80%)
0.50 Physical attractiveness of husbands and wives (75%)
0.40 City size vs incidence of jaywalking (70%)
0.30 Height of husbands and wives (65%)
0.20 IQ vs appreciation of music (60%)
0.05 Earthquakes vs moon position (52.5%)
0.00 Zero accuracy (50% hits as in tossing a coin)
-0.20 Job satisfaction vs absenteeism (40% hits)
-1.00 Perfect inverse accuracy (0% hits = 100% misses)

Effect sizes have many advantages. They are easy to understand. They tell us what we want to know (ie how big the effect is). And we can compare them with effect sizes in other areas to see how they stand up. In what follows we do this for astrology.

Implied effect sizes
You will not find measured effect sizes in astrology books even though their calculation is straightforward. But you will find plenty of implied effect sizes. Thus when astrologers claim that astrology works they are implying that the effect size is usefully more than 0. For example, according to the Faculty of Astrological Studies website, "Astrology provides one of the most powerful and accurate tools for self-knowledge and understanding that we have available", which implies that the effect size is closer to +1 (100% hits, the maximum possible) than to 0 (50% hits, the rate expected by chance). More on this next.

Birth chart of Michael Jackson vs two biographies
To see what the implied effect size might be, let us do what astrologers do -- compare birth charts with their owners. In the figure below we compare the birth chart of Michael Jackson with the superstar himself as described in the biographies of Michael Jackson (MJ) and Jackie Onassis (JO). The better the fit the bigger the implied effect size:

Birth chart of Michael Jackson

A. Jupiter, planet of expansion and success, is close to the MC, indicating career and public image. JO says "He is one of the world's most celebrated entertainers." B. Uranus (innovation) is prominent because close to the Ascendant (outward personality and style). JO says "He is a discoverer of new things, and a fantastic songwriter." C. Saturn (restriction) in secretive Scorpio and the 1st house of self-expression makes him build a wall around himself. MJ says "I've been accused of being obsessed with protecting my privacy. That is true. I'm a person who wishes to be in control of my own life." So far the birth chart is exactly right! D. Sun (inner self) in a benevolent 60-degree angle to Saturn (discipline) makes a person reserved and introverted but also hard-working and ambitious. JO says "For many he seems to be an introverted person." MJ says "To reach the goal I had in mind is beautiful, but to reach more is really beautiful." E. Mercury (communication) is in visionary unconventional Aquarius in the 3rd house of communication. F. Jupiter (expansion) is here ruler of 2nd house (possessions) opposite benevolent Venus, ruler of Taurus (possessions). Both planets confer fame, wealth, and success. The birth chart continues to be exactly right! G. Jupiter is also in perfectionist Virgo, at a 90-degree angle (difficulties) to Mars (energy and action). JO says "He is a perfectionist who continually fights with himself." H. The Moon (sensitivity) is in caring nurturing Cancer. MJ says "Much of my spare time is spent visiting children's hospitals. It brings me great joy to bring joy to those children." Cancer the crab walks sideways. MJ's biography is called Moon Walker. A double whammy! I. Neptune (music, inspiration) in vocal Gemini and 7th house of love and relationships indicates his songwriting skills and their subject matter exactly. Neptune lacks aspects, indicating unrestrained musical freedom. J. Mars with Pluto (intensity, power) are at 90-degree angles (indicating challenges) to Venus (artistic taste) and Jupiter (extravagance). JO says "His way of dancing has a tendency to challenge others." Pluto with Mars (surgery) also rules 12th house (hospitalisation) and its aspects show his plastic surgery and its unhappy consequences. K. Now for the clincher. In 2004 and 2005 progressed Sun is at a 90-degree angle to natal Mars and Pluto in 7th house of relationships, and opposite Jupiter in 9th house of law, exactly indicating his traumatic court prosecution and its timing. Progressed Sun is also conjunct Venus, indicating both the nature of the charges and his eventual acquittal. Everywhere we look the chart is amazingly spot on!

In short, we can see how Michael Jackson's chart confirms what the Faculty claims. The fit between chart and owner is powerful, accurate, and near-perfect. Indeed the chart fits so well, and in such unlikely ways, that coincidence would seem to be ruled out. The effect size in this case would certainly seem to be close to +1 (100% hits). However, there is one small problem, see next.

Seeing faces in clouds
The chart is bogus. The chart we said was Michael Jackson's is actually the chart of Dr Marcel Petiot, one of the worst mass murderers in history. So the powerful and accurate fit with Jackson is meaningless.

Petiot was born on 17 January 1897 with Capricorn Sun, Cancer Moon, Scorpio Ascendant, and was executed in May 1946 for murdering 27 people (he claimed 63). Michael Jackson was born nearly sixty years later on 29 August 1958 with Virgo Sun, Pisces Moon, Gemini Ascendant, and only one aspect in common with Petiot (Venus square Mars but with a much wider orb) -- altogether a very different chart from the one given above which fitted Michael Jackson so exactly.

Of course astrologers could always argue that Jackson and Petiot are actually very similar, the one doing to music what the other did to people. But this powerful and accurate fit to a wildly wrong chart is nothing special. For example in April 1968 the famous French researcher Michel Gauquelin placed this ad in the magazine Ici-Paris offering free personal chart readings:

Gauquelin's ad in Ici-Paris April 1968

To each of over 500 respondents he sent a copy of a 10-page interpretation of Dr Petiot's birth chart pretending it was their own. The interpretation had been generated by an IBM computer programmed by France's leading astrologer Andre Barbault, and said things like "instinctive warmth ... adaptable ... organised ... bathed in a sea of sensitivity ... total devotion to others ... altruistic sacrifices". Gauquelin enclosed an SAE and asked for comments. Of the first 150 replies 94% found the reading to powerfully and accurately fit their character, personal problems, and life events; and 90% found its accuracy to be confirmed by their family and friends. The respondents were generally delighted and impressed by the reading. They said things like "marvellous ... extraordinary ... astonishingly accurate ... certainly me." Remember they all received the same reading.

There are thus two factors that explain why any birth chart will fit any person. (1) The chart's symbolism is so abundant and so ambiguous that we can read into it almost anything after the event, as in the case of Michael Jackson. (2) Interpreting the chart leads to numerous hidden persuaders such as generalities-that-we-see-as-meaningful (the Barnum effect), as in the case of Gauquelin's experiment. Put these two factors together and every chart reading must inevitably persuade astrologers that astrology works. It works because it cannot fail to work even when (as here) the wrong birth chart is used unknown to the participants. In other words the implied effect size has been greatly inflated by factors that have nothing to do with astrology and everything to do with seeing faces in clouds. For more on these factors see Artifacts in reasoning on this website under Doing Scientific Research. But before we look at what the real effect size might be, how do astrologers view this inflation?

How astrologers view inflation of implied effect size
Most astrologers are unaware of it. The rest ignore it or misinterpret it. For example in an article entitled "Astrology as Divination" on the Urania Trust website, British astrologer Geoffrey Cornelius confronts this inflation with the following comment about wrong charts:

"There are few professional astrologers who cannot tell a rueful tale about having the wrong time or even the totally wrong chart for a client and going on to give an unusually effective interpretation or prediction. We are not talking about generalised character readings, either -- these wrong maps can work with high focus symbolism on exact timing and synastry to angles. I have had this experience -- have you?"

Cornelius's views are echoed by Garry Phillipson in his article "Modern Science, Epistemology and Astrology" in Correlation 23(2), 4-23, 2006, where he argues that astrology cannot be understood by science. But as shown above, science allows us to understand astrology (or at least what goes on in consulting rooms) very well. That astrology is not a science does not mean it cannot be understood by science. Phrenology claimed to be a science, but wasn't, and thanks to science we now understand it very well.

Cornelius and Phillipson are suggesting that astrology depends more on some mysterious property of divination than on hidden persuaders. But to psychologists mindful of Occam's Razor ("prefer simple explanations to complex ones") it is the other way round. To them astrology is not a link with the divine but a time-honoured cover for the operation of hidden persuaders. Which leads to the key question: If we control for hidden persuaders, for example by asking astrologers to match charts to owners whose identity is concealed, what is the effect size actually observed? And how does it compare with the effect sizes observed in approaches such as graphology, palmistry, and personality tests?

Observed effect sizes: astrology vs other approaches
The results from hundreds of studies in which hidden persuaders were controlled are summarised in the figure below, which shows the effect sizes actually observed both as correlations and hit rates. On the left are the results for agreement, the extent to which practitioners agree when interpreting the same data. On the right are the results for accuracy, the extent to which the interpretation agrees with reality. Each entry is the mean of all available studies (typically 10-30 studies per mean but can be hundreds per mean). The means can involve correlations calculated in different ways depending on the original study (in technical terms they can be Pearson r's, Spearman rho's, Cohen kappa's, and so on) but for the present purpose the differences are of little consequence. The means can also hide considerable variations between studies but this does not affect the broad indications.

Effect sizes (as correlations and hit rates) for agreement and accuracy

Left. Between practitioners using accepted personality tests and IQ tests, or between repeat tests, the agreement is good. But between astrologers reading the same birth chart there is no useful agreement on what the birth chart indicates. Right. Accepted tests are also more accurate than astrology. Even looking at people's faces (physiognomy, now long discredited) is generally more useful than looking at their birth charts. Reading head bumps (phrenology) hits zero, yet in its heyday its users were convinced it was true for the same reason that users are convinced astrology is true -- by trying it without guarding against hidden persuaders. The Big Arrow shows the mean of 54 studies where a total of 742 astrologers read a total of 1407 charts for personality, aptitude, or life events, with a mean accuracy of r = 0.034, equivalent to a hit rate of 51.7% vs 50% expected by chance.

The message of the above survey is very simple. It says that effect sizes in astrology, whether for agreement or accuracy, when controlled for hidden persuaders and other non-astrological effects, are not usefully better than tossing a coin. In other words astrology is neither powerful nor accurate. Accepted approaches are better, usually much better. As with the Emperor's New Clothes, astrology's implied performance has disappeared on critical inspection. Especially telling is the negligible agreement between astrologers (mean r = 0.098, equivalent to a hit rate of 54.9% vs 50% expected by chance), which is based on 28 studies and can therefore hardly be dismissed as untypical. It is telling because (1) only agreement is relevant, not accuracy, so the studies cannot be dismissed on the grounds of poor material, and (2) astrology seems pointless if astrologers cannot usefully agree on what a given birth chart means. On the other hand, there is more to astrology than having a useful effect size, see next.

Detail from title page of Robert Fludd's Utriusque Cosmi ... Historia 1716

As art, as religion, astrology delivers
Serious astrology offers symbols, metaphors, and images of undeniable appeal, which can feed the inner person in the same way as music, poetry and art do. It provides emotional comfort, spiritual support, and interesting ideas to stimulate self-examination. Good things to have in our dehumanised society.

In this sense the birth chart is also an art form. Its images can help some people to make connections, to link experiences, to understand (or at least seem to understand) cause and effect, and ultimately to understand their place within the whole, all of which is about belonging. Such people respond to astrology probably because it seems to have its priorities right. The same applies to religion, spiritual and new age philosophy, and pop psychology in all their different forms, even though they often disagree and therefore cannot all be right.

In this way astrology, although not usefully accurate, may still for some people be a powerful benefit. The problem of course is that people who derive benefit from astrology are not going to believe evidence for its inaccuracy. To them it feels good therefore the facts are irrelevant. If we are careful to see accuracy and benefit as two separate things, we can avoid making the same mistake. For more on how reverting to a declared art form may be the only valid way ahead for astrology, see Optimum place for astrology on this website under Applied Astrology.

Statue of Liberty at the entrance to New York harbour

Postscript
Powerful and accurate fits are normal in astrology even when the subject is inanimate. For example if we compare the Statue of Liberty, the USA's most famous landmark, with the most dominant feature in the chart of the USA (Sun conjunct Uranus), we find an innovative (Uranus) national monument (Sun) resting on granite (ruled by the Sun), lit by electricity (Uranus), 151 feet high (= 1 Leo, ruled by the Sun), 305 feet above ground level (= 5 Aquarius, ruled by Uranus), on an 11-pointed island (Aquarius, the 11th sign), bearing an inspiring (Uranus) poem "The New Colossus" (Sun) with lines such as "sunset gates ... golden door" (Sun) and the self-willed "Keep ancient lands, your storied pomp!" (Uranus). When it was declared a National Monument on 15 October 1924, the Sun was in Venus-ruled Libra (bronze), Uranus was in Neptune-ruled Pisces (New York harbour), and Neptune was at the midpoint of Sun-Uranus. And of course the Statue symbolises to newcomers the freedom (Uranus) and opportunity (Sun) offered by the USA. Once again the fit is exactly right in many amazing ways and could hardly be more impressive. But once again the chart is bogus! There are no Sun-Uranus contacts in the US charts that are generally accepted (Campion N, The Book of World Horoscopes, Aquarian, 1988, pages 293-303). Once again we are making matches after the event and seeing faces in clouds.

From www.astrology-and-science.com         Click here for home page or fast-find index