🕹 Eating Mangoes with Hamlet
Hamlet is also, yeah, I mean,… like us! Random!!
Introduction
Let us read the famous nunnery scene from Hamlet: To be or Not to be…
| Shakespeare | Plain English (hopefully) |
|---|---|
| To be, or not to be: that is the question: | The first six words of the soliloquy establish a balance. There is a direct opposition – to be, or not to be. Hamlet is thinking about life and death and pondering a state of being versus a state of not being – being alive and being dead. |
| Whether ’tis nobler in the
mind to suffer The slings and arrows of outrageous fortune, Or to take arms against a sea of troubles, And by opposing end them? |
The balance continues with a consideration of the way one deals with life and death. Life is a lack of power: the living are at the mercy of the blows of outrageous fortune. Should we nobly suffer or should we take arms and fight against these random things? |
| To die: to sleep; No more; and by a sleep to say we end The heart-ache and the thousand natural shocks That flesh is heir to, ’tis a consummation Devoutly to be wish’d. |
The only action one can take against the things he lists among those blows is to end one’s life. That’s the only way of opposing them. The ‘sleep of death’ is therefore empowering: killing oneself is a way of taking action, taking up arms, opposing and defeating the slings and arrows of outrageous fortune. Living is a passive state; dying is an active state. |
| To die, to sleep; To sleep: perchance to dream: ay, there’s the rub; For in that sleep of death what dreams may come When we have shuffled off this mortal coil, Must give us pause: there’s the respect That makes calamity of so long life; For who would bear the whips and scorns of time, The oppressor’s wrong, the proud man’s contumely, The pangs of despised love, the law’s delay, The insolence of office and the spurns That patient merit of the unworthy takes, When he himself might his quietus make With a bare bodkin? who would fardels bear, To grunt and sweat under a weary life, |
Death is something desirable – devoutly to be wished, a consummation – a perfect closure. It’s nothing more than a sleep. But there’s a catch, which Hamlet calls a rub. A ‘rub’ is a bowls term meaning an obstacle on the bowls lawn that diverts the bowl, so the fear of the life hereafter is the obstacle that makes us pause and perhaps change the direction of our thinking. We don’t control our dreams so what dreams may come in that sleep in which we have shuffled off all the fuss and bother of life? He uses the term ‘mortal coi l,’ which is an Elizabethan word for a big fuss. With that thought, Hamlet stops to reconsider. What will happen when we have discarded all the hustle and bustle of life? |
| But that the dread of
something after death, The undiscover’d country from whose bourn No traveller returns, puzzles the will And makes us rather bear those ills we have Than fly to others that we know not of? |
The problem with the proposition is that the sleep of death is unknown and could be worse than life. Hamlet now lets his imagination wander on the subject of the voyages of discovery and the exploratory expeditions. Dying is like crossing the border between known and unknown geography. One is likely to be lost in that unmapped place, from which one would never return. The implication is that there may be unimagined horrors in that land. |
Thus conscience does make
cowards of us all;
|
But there is more to it than that. It is not just about killing himself but also about the mission he is on – to avenge his father’s death by killing his father’s murderer. Throughout the action of the play, he makes excuses for not killing him and turns away when he has the chance. ‘Conscience does make cowards of us all.’ Convention demands that he kill Claudius but murder is a sin and that conflict is the core of the play. At the end of the soliloquy, he pulls himself out of this reflective mode by deciding that too much thinking about it is the thing that will prevent the action he has to rise to. |
| –Soft you now! The fair
Ophelia! Nymph, in thy orisons Be all my sins remember’d. |
Do you see a lot of “randomness” in that famous soliloquy? What do you reckon was Hamlet’s problem? In this soliloquy, Hamlet gives a list of all the random things that annoy him about life: the whips and scorns of time, the oppressor’s wrong, the proud man’s contumely, the pangs of despised love, the law’s delay, the insolence of office and the spurns that patient merit of the unworthy takes. There’s a sense of agonized frustration in this soliloquy that however bad life is we’re prevented from doing anything about it by fear of the unknown.
And yet, being Prince of Denmark, and having a lot of Questions, and unmistakable Authority, he has to Act! How did he solve that problem? We don’t know for sure if Hamlet believed in Goddess Saraswati, Shakespeare doesn’t say, but Christopher Isherwood seems to think so, on Page 3.
Aside: in Hindu mythology, who are the two who went into Hamlet’s “the undiscover’d country” and did return? She and he. But how did they manage it?
Yes, they asked Questions!
Suppose we were Hamlet….
What would we want to know? What Questions would we have?
- Who is our friend?
- What is the chance that we will survive?
- Who killed Dad?
Did Hamlet have an idea, any suspicion, that he might have wanted to prove? Yes, he did, and that is why he feigned madness, so people would open up and tell him things, in an unguarded way. So there was craft in his madness!!
So here Hamlet tells his friends, Horatio and Marcellus, what he is going to do, and swears them into silence:
Here as before. [For the third time, Hamlet places his sword for the others to swear upon]
Never, so help you mercy,
How strange or odd soe’er I bear myself
(As I perchance hereafter shall think meet
To put an antic disposition on)
That you at such time seeing me never shall,
With arms encumbered thus, or thus head shaked,
Or by pronouncing of some doubtful phrase
As, “Well, we know,” or “We could an if we would,”
And especially don’t say Hakuna Matata, Dude!!
A Hypothesis worthy of Hamlet
So we know what Hamlet wanted to know, and how he planned to find out.
What about us?
If we wanted to know the truth of something, some guess, a hunch, or a Hypothesis that we had, then we would need to conduct an “Experiment”, rather like Hamlet did, without quite going or feigning madness of course. But there is a Madness to/in the Method, as we will see…
From StatsRef :
The PPDAC summary table suggests a relatively linear flow from problem definition/Hypothesis through to conclusions — this is typically not the case. It is often better to see the process as cyclical, with a series of feedback loops. A summary of a revised PPDAC approach is shown in the diagram below. As can be seen, although the clockwise sequence (1→5) applies as the principal flow, each stage may and often will feed back to the previous stage.
In addition, it may well be beneficial to examine the process in the reverse direction, starting with Problem definition and then examining expectations as to the format and structure of the Conclusions (without pre-judging the outcomes!). This procedure then continues, step-by-step, in an anti-clockwise manner (e→a) determining the implications of these expectations for each stage of the process.
Design of Experiments
Let us pretend we are Hamlet and design a simple experiment to find out some “truth” (highly probable idea) that is important to us.
No Free Hunch: Hypothesis Design
We learnt a lot (new) words from Hamlet, didn’t we? How many do we remember? What Hypothesis could be make? What do we have a hunch about?
Let us consider the Short Term Memory(STM) of Foundation Students at SMI as our vehicle for testing similar Hypotheses.1
- STM scores are significantly different between students of Design and those of Art (;-O)
- STM is much better for those who use Windows as compared to those who use Macs
- Any other factors, not related to types of individuals, but more
like parameters of STM itself, also affect the STM scores, for
example:
- Word complexity (Syllables per Word)
- No of Words
- Exposure Time and Recall Time
With the Guilford Divergent Thinking Test, you might consider:
- Students who speak their Mother Tongue fluently score better in Divergent Thinking Scores (Fluency, Flexibility, Elaboration, Originality)
- There is a significant difference in Guilford Scores between students from “small towns” and those from metros.
Factors in Hypothesis
In each of these Hypotheses, we can see that there are binary (Yes/No)
(two-valued) parameters that are part of the test: Art vs Design,
Windows vs Mac, Speaks Mother Tongue or Not, Small Town vs Metro,
Long vs Short List of Words, Short vs Long Words and
Long vs Short Exposure.
If we want our experiment to be fair we must have:
- Devise a test tool for each combination of binary parameters (Test Type)
- the same number of subjects for each Test Type
- randomly select the people as respondents each Test Type
We will choose one of these questions, or a similar one, and use the PPDAC Method to proceed with our Experiment. This paper by Lawrence to guide our Hypothesis Testing and the Design of our Experiment:
Lawrence - A Design of Experiments Workshop as an Introduction to Statistics - PDF).
This is the paper describing a simple Design of Experiments workshop in class much like ours. We will try to mimic as much of this as we can. Do read through this in the evening today in preparation for our experiment.
Data Analysis-1: Comparison of STM Histograms using Excel and WTFcsv
Once our test/survey is complete, let us collate all the data for all tests using this Excel spreadsheet, which we can place on Google Drive for the whole class. (This is for the STM Hypothesis; we can adapt this spreadsheet for any other hypothesis test with categorical factors andscores.)
- Enter the data from all tests on the first sheet titled “RawData” (the one with the warning)
- Download and save as your own named local copy.
- Duplicate this data on the sheet titled “Data Duplicated”. Do not touch the “RawData” sheet again. Good Practice to not touch original data!!
- For each of the factors under consideration, we will need to stack up STM scores from \(2^3 = 8\) tests, i.e. eight columns.
- If we have three two-valued, categorical factors, we will get three sets of stacked scores, each from a (shuffled) set of the tests.
- Half the scores in these stacked columns will pertain to one level of the factor, and the other half of the scores will pertain to the other level. (This should remind you of the Karnaugh Map you may have learnt in your digital logic courses in school.). See example below: it shows how we can stack up the scores from 4 tests for each level of the parameter p1.
| scores | test | p1 | p2 | p3 |
|---|---|---|---|---|
| 20 | A | True | Long | Big |
| 20 | B | True | Long | Small |
| 20 | C | True | Short | Big |
| 20 | D | True | Short | Small |
| 20 | E | False | Long | Big |
| 20 | F | False | Long | Small |
| 20 | G | False | Short | Big |
| 20 | H | False | Short | Small |
- We will set up each of these two stacked-score columns in Col A on separate sheets, similar to the one shown in the sheet titled “Permutation Test”. One sheet for each two-valued parameter.
- Create paired Histograms for each of the two stacked-score columns using WTFcsv, or Excel itself if you are confident. Ensure that there are TWO Histograms: one for each value of the factor under consideration.
- Inspection of the shapes and locations of these paired Histograms may give you an idea whether the factor under consideration has any effect on STM scores or none.
- Save these 6 histograms as PNG files on your machines and use them along with Comic Generator (discussed below) to tell the story of your Hypothesis Testing.
Computing Parallel Worlds: The Permutation Test
BTW, why did Gabbar Singh say to Kaalia and the others in Sholay, “Hum ko kuchh nahi pata”? And then afterwards, “Kamal ho gaya”…why does he say that?
What if we cannot trust our eyes to compare the pair-wise Histograms? If there was considerable overlap? Is there a better way? Can we fix this by playing another Childhood Game? Yes, with a deck of cards! But before we play with cards, some background:
From:
Gabbar SinghTim C. Hesterberg (2015) What Teachers Should Know About the Bootstrap: Resampling in the Undergraduate Statistics Curriculum, The American Statistician, 69:4, 371-386 DOI: 10.1080/00031305.2015.1089789
Student B. R. was annoyed by TV commercials. He suspected that there were more commercials in the “basic” TV channels, the ones that come with a cable TV subscription, than in the “extended” channels you pay extra for. To check this, he collected the data shown in Table 1. He measured an average of 9.21 minutes of commercials per half hour in the basic channels, vs only 6.87 minutes in the extended channels. This seems to support his hypothesis. But there is not much data—perhaps the difference was just random. The poor guy could only stand to watch 20 random half hours of TV. Actually, he didn’t even do that—he got his girlfriend to watch half of it. (Are you as appalled by the deluge of commercials as I am? This is per half-hour!)
| basic | extended |
|---|---|
| 6.950 | 3.383 |
| 10.013 | 7.800 |
| 10.620 | 9.416 |
| 10.150 | 4.660 |
| 8.583 | 5.360 |
| 7.620 | 7.630 |
| 8.233 | 4.950 |
| 10.350 | 8.013 |
| 11.016 | 7.800 |
| 8.516 | 9.580 |
## [1] 2.3459The average difference in ad times between the two sets of TV channels is 2.34.
How easy would it be for a difference of 2.34 minutes to occur just by chance? To answer this, we suppose there really is no difference between the two groups, that “basic” and “extended” are just labels. So what would happen if we assign labels randomly? How often would a difference like 2.34 occur?
We’ll pool all twenty observations, randomly pick 10 of them to label “basic” and label the rest “extended”, and compute the difference in means between the two groups. We’ll repeat that many times, say ten thousand, to get the permutation distribution shown in Figure 1.
| channel | times |
|---|---|
| basic | 6.950 |
| extended | 3.383 |
| basic | 10.013 |
| extended | 7.800 |
| basic | 10.620 |
| extended | 9.416 |
| basic | 10.150 |
| extended | 4.660 |
| basic | 8.583 |
| extended | 5.360 |
| basic | 7.620 |
| extended | 7.630 |
| basic | 8.233 |
| extended | 4.950 |
| basic | 10.350 |
| extended | 8.013 |
| basic | 11.016 |
| extended | 7.800 |
| basic | 8.516 |
| extended | 9.580 |
The observed statistic 2.34 is also shown; the fraction of the distribution to the right of that value (≥ 2.34) is the probability that random labeling would give a difference that large. In this case, the probability of this value ( area coloured in green ) is < 0.005.
It would be rare for a difference this large to occur by chance. We have randomly tried “all possible chances” and are hardly able to achieve similar, and the rarer something is, the more likely that there is an underlying truth.
And therefore we conclude there is a real difference between the groups and that ad time is different between
basicandextendedTV channels.
Data Analysis-2: Creating Parallel Worlds
- We will execute on Permutation Test on one sheet step by step and decide whether that factor had a significant effect on STM scores.
- We pretend that there is no difference in scores whether the factor is chosen on way or another.
- We mechanize this pretence by lumping “both kinds” of scores together and shuffle them and divide them, randomly into two groups, and take the difference in scores.
- We can do this, like we did with our Monte Carlo experiment, many many times and calculate difference in scores each time. (It is like inventing many parallel worlds)
- If we look at the way these randomly computed scores are distributed and compare with the one measurement we did see, we can decide whether Mother Nature is up to something, or we are able to mimic the Mom.
- If we are not able to mimic Mom, then Mom always knows and we bow to her and ascribe a significance to the factor under consideration. Else, nope.
- Replicate this test for the other (binary) factors.
Apropos: how could we do this for non-binary factors …??!!!
Making a Data Comic out of our Experiment
What have been our Conclusions with the Experiment? Let us take our Experiment and make a comic out of it: Data Comic Generator from Gramener (Weblink)
References
- Randomized Trials:
A. J. Lawrance (1996) A Design of Experiments Workshop as an Introduction to Statistics, The American Statistician, 50:2, 156-158, DOI: 10.1080/00031305.1996.10474364. PDF
Victoria Woodard (2023) Five Hands-on Experiments for a Design of Experiments Course, Journal of Statistics and Data Science Education, 31:3, 225-235, DOI: 10.1080/26939169.2023.2195889. PDF
Tim C. Hesterberg (2015). What Teachers Should Know About the Bootstrap: Resampling in the Undergraduate Statistics Curriculum, The American Statistician, 69:4, 371-386, DOI:10.1080/00031305.2015.1089789. PDF
Shuttleworth, M. (2009). What is the scientific method? Retrieved from https://explorable.com/what-is-the-scientific-method
Mahashweta Devi, The Why Why Girl, https://www.tulikabooks.com/picture-books/the-why-why-girl-english.html, read by Sarah Dryden Peterson https://www.youtube.com/watch?v=xNcV5DZhg4c
Could this song indicate what our situation might be?
Fun Stuff: Stat Lessons from Sholay
Gabbar: “Kitne Aadmi thay?
Stats Teacher: How many observations do you have? n < 30 is a joke.
Gabbar: Kya Samajh kar aaye thay? Gabbar khus hoga? Sabaasi dega kya?
Stats Teacher: What are the levels in your Factors? Are they binary? Don’t do ANOVA just yet!
Gabbar: (Fires off three rounds ) Haan, ab theek hai!
Stats Teacher: Yes, now the dataset is balanced wrt the factor (Treatment and Control).
Gabbar: Is pistol mein teen zindagi aur teen maut bandh hai. Dekhte hain kisko kya milega.
Stats Teacher: This is our Research Question, for which we will Design an Experiment.
Gabbar: (Twirls the chambers of his revolver) “Hume kuchh nahi pataa!”
Stats Teacher: Let us perform a non-parametric Permutation Test for this Factor!
Gabbar: “Kamaal ho gaya!”
Stats Teacher: Fantastic! Our p-value is so small that we can reject the NULL Hypothesis!!
Go and like this post at: https://www.linkedin.com/pulse/stat-lessons-from-sholay-arvind-venkatadri-wgtrf/?trackingId=c0b4UCTLRea6U%2Bj%2Bm4TCtw%3D%3D
More Fun Stuff: Words Belong Hamlet
Hamlet’s To be or not to be in Pidgin English Tok Pisin!
Which way this time? Me killem de finish body b’long me. Or me no do ’im? Me no savvy.
Might ’e better ’long you-me catchem this fella, string for throw ’im this fella arrow.
Altogether b’long number one bad fella, name b’long him fortune? Me no savvy.
Might ’e better ’long you-me For fightem ’long altogether where him ’e makem you-me sorry too much.
Bimeby him fall down die finish? Me no savvy.
Or we could use Guilford Alternative Uses Creative Test↩︎