I have started out with this broad division of measurements into continuous and categorical variables. The skill of identifying whether your variables are continuous or categorical is absolutely essential for deciding what statistical test to use, and in general the division between continuous and categorical is the most important division to know when it comes to performing statistical tests, as some tests can be used only when all variables are continuous, while other tests can be used only when there is at least one categorical and one continuous variable.
However, many authors make a more detailed division of variable measurement into three or even four different types of measurement. For example, Mackey and Gass discuss three levels of measurement: nominal, ordinal, and interval scales. Nominal scales which measure categorical variables do not have any inherent numerical value. Ordinal scales are rank-ordered and do have some inherent numerical value.
For example, we might have first-, second-, and third-year students of French. If we assigned the students a number based on their number of years of study, we would expect this to be reflected in their knowledge of French, with students with a higher number having more knowledge of French. We might not expect, however, that the difference between third-year students and second-year students would be the same as between secondyear and first-year students.
In other words, there is no guarantee that there are equal intervals of measurement between the participants. In an interval scale which measures continuous variables , however, we would like to say that the difference between getting a 9 and a 10 on a point test is the same as the difference between getting a 3 and a 4 on the test. In other words, the intervals are assumed to be equal. Some authors will also distinguish a ratio scale Howell, , which has all the properties of an interval scale plus the scale has a true zero point.
I can t think of any. A lb rock is twice as heavy as a lb rock, and something that weighed 0 lb means that the thing that is being measured is absent, so the scale truly has a zero point. The truth is, you probably don t, but it is good to be familiar with the terms Howell, , agrees with me on this!
As I noted above, the division between categorical and continuous variables which is the same as the division between nominal and interval scales is in practice the division which is important in deciding which statistical test to choose. Researchers who use a measurement which is ordinal end up classifying it as either categorical or continuous, often depending on how many points there are in the scale. For example, if you have recorded the years of study of college learners of French and you have three levels, you will most likely call this a categorical variable with no inherent numerical value, even though you do realize that most learners in their third year of French study will perform better than learners in their first year.
On the other hand, if you have a seven-point Likert scale measuring amount of language anxiety, you will be likely to call this a continuous measurement, even though you may acknowledge that a score of 4 on the scale moderate anxiety may not be exactly twice of what the score of 2 is little anxiety. In truth, it is not the measurement itself but how the researcher chooses to look at it that is the deciding factor in whether you will call a measurement continuous or categorical.
For example, you may ask people their age, which is certainly an interval scale in fact, it is a ratio scale, because it has a true zero point , but then classify them as young versus old, making your measurement into a nominal scale. This example shows what Howell states, which is that the researcher must remember that the event itself and the numbers that we measure are different things, and that the validity of statements about the objects or events that we think we are measuring hinges primarily on our knowledge of those objects or events, not on the measurement scale p.
The variables measured in the study to answer the research question are listed in the tables. Look at each variable and decide whether it is categorical or continuous. Within the category of interval scale that I have outlined here, Crawley distinguishes several more levels that can affect the choice of statistical test. Thus, for a dependent variable which a beginner might treat as an interval scale, we can distinguish between those measurements which are proportions or counts. If the measurement is a proportion such as the number of mistakes out of every words then a logistic model would provide a better fit than a normal regression or ANOVA model.
If the measurement is a count such as the number of mistakes on a writing assignment then a log linear model provides a better fit. And if a dependent variable which is nominal has only two choices and is thus a binary scale, a binary logistic model provides a better fit.
Accuracy level measured on a series of different tests, but generally most of the tests had about 16 points or more. Fluency ratings done by judges who listened to taped conversations and rated them for fluency on a point scale. Number of phrases the child knows based on the MacArthur Inventory parental report. Number of phoneme errors on 96 nonwords. Both independent and dependent variables can be measured as nominal, ordinal, or interval, so this division is independent of the measurement scale.
This dichotomy relates to the way the variables function in the experiment. Independent variables are those that we think may have an effect on the dependent variable. Often the researcher controls these variables. For example, an independent variable would be membership in one of three experimental conditions for learning vocabulary: mnemonic memorization, rote memorization, and exposure to context-laden sentences. The researcher randomly assigns participants to the groups, and suspects that the group the participant is in will affect their scores on a later vocabulary test.
Independent variables are not always variables that the researcher manipulates, however. If you think that amount of motivation and presence at a weekly French conversation chat group influence fluency in French, you will simply try to measure motivation and chat group attendance.
In this case your measure of motivation and a count of number of French club meetings attended are your independent variables. You have not fixed them in the way you might assign students to a certain group, but you are still interested in how these variables influence fluency. Here are some common independent variables in second language research studies: experimental group membership L1 background proficiency level age status as monolingual or bilingual type of phonemic contrast amount of motivation or language anxiety amount of language use The variable that you are interested in seeing affected is the dependent variable.
For our French example, the dependent variable is a fluency in French measure. If the independent variables have been fixed, then the dependent variable is beyond the experimenter s control. For example, the researcher may fix which experimental group the participants belong to, and then measure their scores after the treatment on a vocabulary test. Assignment to a group has been fixed, but then scores on the vocabulary test are out of the control of the researcher. Of course, the researcher is hoping betting?
In this sense, the dependent variable depends on the independent variable. Here are some common dependent variables in second language research studies:. I remember first encountering the idea of dependent and independent variables, and it took a little while before I got them straight, but maybe it will help if you look at a variable and think, Do I think this data will be influenced by some other variables?
If so, it is a dependent variable. If you think the data will be an explanatory factor for some other results, then it is an independent variable. While dependent variables may be categorical or continuous, independent variables are generally categorical. It is extremely important to understand the difference between independent and dependent variables correctly because this classification will affect what test you will choose to analyze your data with.
Now I hate to tell you this, but actually the difference between independent and dependent variables is not always clear cut, and in some cases this traditional dichotomy may not really be applicable. In this discussion I have been assuming that you will have a variable that depends for its outcome on some other variables the independent variables.
In such a case the researcher is clearly trying to establish some kind of causal relationship. In other words, I cause outcomes to differ because I put the participants in different experimental groups, or I cause reaction times to differ because some participants had to answer questions with syntactical violations. However, sometimes you will not find a cause-and-effect relationship. You will only be looking at the relationship between two variables and you would not be able to say which one depends on the other.
For example, if you want to know how time spent in the classroom per day and time spent speaking to native speakers are related, you would be interested in the relationship between these two variables, but you could not say that time spent in the classroom necessarily affected the time students spent speaking to native speakers of the language, or vice versa.
In this case, correlation is the type of test you would look at to examine whether there are important relationships between the variables. Robinson has a research question where he was not looking for a cause-and-effect relationship between the variables. His question was whether there was a relationship between measured intelligence scores and the ability to abstract rules of language implicitly without conscious awareness. He thus used a correlation to examine the possible relationship between intelligence and implicit induction ability.
The next section gives some practice in identifying dependent and independent variables in experimental studies. All of the research questions given here are from real studies, but for the sake of simplicity sometimes they have been modified so that they focus on one specific aspect of the entire study. But be careful there may be. Application Activity: Practice in Identifying Variables Write the independent and dependent variables in the tables.
In some cases, there may be more than one independent variable, but there is never more than one dependent variable. Independent: Dependent: 2 Munro, Derwing, and Morton : Does L1 background influence the way that English learners rate the speech of other English learners for comprehensibility? Independent: Dependent: 3 Hirata : Does pronunciation training with visual feedback help learners improve their perception of phonemically contrastive pairs of words more than training without visual feedback?
Independent: Dependent: 4 Proctor, August, Carlo, and Snow : What kind of relationship exists among an oral language measure and a reading comprehension measure in English for fourthgrade bilinguals? Independent: Dependent: 5 Bialystok, Craik, Klein, and Viswanathan : Do age and being bilingual affect the accuracy with which participants perform on the Simon test a test that measures whether participants can ignore irrelevant information? Independent: Dependent: 6 Wartenburger et al.
Independent: Dependent: 8 Larson-Hall : Do learners perform differently in their ability to perceive phonemic contrasts in nonwords in the second language depending on their proficiency level and on the sounds that are contrasted? Independent: Dependent: Summary of Variables You should now be able to take a research question and decide what the variables are, whether they are categorical or continuous variables, and whether they are dependent or independent variables.
Once you can take this step, you will be very far along on the path toward deciding what type of statistical test you ll need to analyze your data. For example, if you know that you have two continuous variables and that you are examining a relationship between the variables and there is thus no independent or dependent variable then you ll want to use the statistical technique of correlation.
On the other hand, if you have a research question that has a dependent and an independent variable, and the dependent variable is continuous while the independent variable is categorical with only two groups, then you know you ll want to choose a t-test. We ll leave more of this type of fun until later won t it make you feel smart to know what type of test you want? A research model that contains both fixed and random effects is called a mixed-effects model, and this is not treated in this book.
However, there may be times when it is important to understand the difference between fixed and random effects, so I will include that discussion here. Fixed effects are those whose parameters are fixed and are the only ones we want to consider.
Random effects are those effects where we want to generalize beyond the parameters that make up the variable. A subject term is clearly a random effect, because we want to generalize the results of our study beyond those particular indi-. If subject were a fixed effect, that would mean we were truly only interested in the behavior of those particular people in our study, and no one else.
Note that the difference between fixed and random factors is not the same as between-subject and within-subject factors. Table 2. This table draws upon information from Crawley , Galwey , and Pinheiro and Bates The researcher presents basic information about participants such as their age, gender, first language, and proficiency level in a second language, and then gives mean scores on whatever test was used.
The researcher then reports something called a p-value and proclaims whether the results are statistically significant or not. If the results are statistically significant, the researcher assumes the hypothesis was correct and goes on to list implications from this finding. If the results are not statistically significant, the researcher tries to explain what other factors might need to be investigated next time.
This impression of how statistical testing works is often not mistaken on the surface this is indeed what you often read in the research paper , but there is a variety of issues Table 2. If one of the levels of a variable were replaced by another level, the study would be radically altered.
Fixed effects have factor levels that exhaust the possibilities. We are only interested in the levels that are in our study, and we don t want to generalize further. Fixed effects are associated with an entire population or certain repeatable levels of experimental factors. Random effects Random effects have uninformative factor levels. If one of the levels of a variable were replaced by another level, the study would be essentially unchanged.
Random effects have factor levels that do not exhaust the possibilities. We want to generalize beyond the levels that we currently have. Random effects are associated with individual experimental units drawn at random from a population.
Examples of fixed effects: treatment type male or female native speaker or not child versus adult first language L1 target language Examples of random effects: subjects words or sentences used classroom school.
It is to these hidden assumptions that we will turn our attention in this section Hypothesis Testing In a research paper, the author usually provides a research question, either directly or indirectly. As a novice reader, you will think that this hypothesis, based on previous research and logical reasoning, is what is tested statistically. You are wrong. It turns out that a statistical test will not test the logical, intuitively understandable hypothesis that you see in the paper but, instead, it will test a null hypothesis, which is rarely one the researcher believes is true.
This topic will be treated in more detail in Chapter 4, but it is very confusing to new researchers, so I will introduce it briefly in this chapter. To conduct null hypothesis testing, we will set up a null hypothesis H 0 that says that there is no difference between groups or that there is no relationship between variables. It may seem strange to set up a hypothesis that we may not really believe, but the idea is that we can never prove something to be true, but we can prove something to be false Howell, , p.
Actually, this is a good point to remember for the whole enterprise of scientific hypothesis testing in a quantitative approach one that deals with numbers, thinks that truth can be ascertained objectively, and seeks to have replicable and verifiable results.
You should be careful when you make claims that a study has proved a point, because we can t prove something to be true! Instead, say that a study supports a certain hypothesis or provides more evidence that an idea is plausible. The old chestnut says that you cannot prove that unicorns don t exist; you can only gather data that show it is likely that they don t.
On the other hand, it will only take one unicorn to prove the hypothesis false. Let s walk through the steps of this with our researcher. Let s say our researcher believes that pre-class massages will decrease language learning anxiety a unique idea that I have not seen in the literature, but I m sure I d like to be a participant in the study!
In the research report, the researcher lays out the logical hypothesis: H logical : Pre-class massages will decrease language learning anxiety, so that a group that gets pre-class massages will have less anxiety than a class that does not. However, the researcher has also somewhere, perhaps only in their head, laid out a null hypothesis: H 0 : There is no difference in language learning anxiety between the class that gets pre-class massages and the class that does not.
See Streiner for more information. This is our one unicorn we needed to prove the hypothesis false! Now just because we have proved the hypothesis false does not mean that we have proved the opposite or the logical hypothesis is true. But what we saw is that we can reject the null hypothesis. Now, if we reject the null hypothesis, this means we reject the idea that there is no difference between groups or no relationship among variables.
Once we reject the null hypothesis we will be able to accept the alternative hypothesis, which in this case would be: H a : There is a difference in language learning anxiety between the class that gets pre-class massages and the class that does not. If we consider an extremely simplified example relating to groups we can see that mathematically it is easier to prove the null hypothesis false than to prove it true: H o : The difference between the groups is 0 no difference between groups.
H a : The difference between the groups is not 0 a difference between groups. Let s say that we then calculated the average score for each group and found that the difference between groups was only Although this seems like a very small difference, it would refute the null hypothesis that the difference was zero it would be our unicorn.
On the other hand, to prove the alternative hypothesis, we could gather lots of examples of times when the difference between groups was not zero, but this would not prove that the difference was zero in the same way that collecting lots of evidence that horses but not unicorns exist would not prove that unicorns do not exist.
Although hopefully I ve convinced you here that a null hypothesis is much easier to prove than the logical reasoned hypothesis you might really set out to investigate, there are problems with using this system of null hypothesis testing. We ll examine the problems with this approach and how to best remedy them in more detail in Chapter Application Activity: Creating Null Hypotheses Look at the research questions given below and create a null hypothesis: 1 Boers, Eyckmans, Kappel, Stengers, and Demecheleer : Will explicit instruction in noticing formulaic sequences help one group speak more fluently than another group that does not get this instruction?
Populations versus Samples and Inferential Statistics The next assumption you may have about how statistics works is that a researcher tests a group of people and these are the people that the researcher wants to comment about. You may think this because the researcher presents summary information about the participants such as their mean age, the number of males and females tested, their L1 backgrounds, and so on.
In most cases, you are wrong. The researcher must indeed test a certain selection of people, called a sample. But the researcher in fact does not want to comment just about these people, but instead wants to make a generalization to a wider group of people.
If the researcher were really going to comment only on whether brain activation patterns from an fmri differentiated eight bilingual participants from eight monolingual participants in Dresden, Germany, you probably wouldn t care much. Usually you will care because of what you think the research study says about differences between bilinguals and monolinguals in general and in a variety of languages.
What you care about is the wider population of people that the sample was drawn from. The population is the group of people or collection of texts or test scores in which you are interested. Many second language research studies test a sample of only 10 or 15 participants in each group, but they are of interest to the wider research community because of the population that readers suppose the findings apply to. The population may be assumed to be all English as a second language learners in the United States, or all French as a foreign language learners in England, or all immigrants of all language backgrounds to any country.
Put this way, it may sound a little grandiose to think that our studies, done with a very small sample, apply to such large populations! The reason that we may be able to make the leap from conclusions drawn on a sample to conclusions drawn to the wider population is because of inferential statistics. Inferential statistics are those that can be used to make inferences to the population that the sample is assumed to come from. Inferential statistics can be contrasted with descriptive statistics, which are measures that are derived from the sample.
So when you read a research report and see the number of males and females who participated, and their mean ages and their mean scores on a test, these numbers are all descriptive statistics. Descriptive statistics may include counts of how many times some feature is found in a piece of writing, or graphs such as histograms which graphically show such counts. Descriptive statistics are very appropriate in situations where you want to numerically describe some kind of phenomenon.
More information about how to calculate descriptive statistics is found in the following chapter. You will definitely want to include descriptive statistics in your research report; conversely, if you are reading a research report you should understand what an average score means. But usually a research report will not stop with descriptive statistics. If a statistical test is conducted, then the researcher assumes that the results of that test can be applied to the wider population from which the sample was selected unless they explicitly note some reservations.
One of the problems with making this kind of inference, however, is that, for inferential statistics to be applicable, the sample should be a random sample. This means that each person or every thing in the population is equally likely to be picked. When a second language research study is conducted with intact classrooms, although this is convenient for the researcher, the sampling is not.
Maybe students have heard about the teachers and some have deliberately chosen a teacher they think they will like; maybe most of the students who are taking the 8 a. Spanish class are business majors who all have required classes at the other times that Spanish is offered. There are any number of factors that may make the intact classroom less than a random sampling of students.
An experiment that uses volunteers is also not a random sampling because those who volunteer to do something are often different from those who would never voluntarily include themselves in a study. Hatch and Lazaraton note that a pool of volunteers is a group from which a random sample might be drawn, although the volunteers still may not be representative of the population.
To imagine a truly random sample, think of a political survey you might hear about on the news that says how many people will vote for the Republican candidate this election year and how many are voting for the Democratic candidate. Such surveys do not sample every registered voter in the country; however, they do identify registered voters in a certain area as their population and then contact a random sampling of those voters until they reach a certain number of participants that has been decided on beforehand they test about 1,, I heard John Zogby, a pollster, say on the Diane Rehm show on September 3, With such a random sampling, pollsters can then perform inferential statistics on the data and say that they would expect the same kinds of results even with different samples of people within certain boundaries of error.
They thus try to generalize the results from a particular sample of people to a wider population. Inferential statistics try to say something about the overall population of people or data that they describe, without sampling every piece of data or every person in that set. As you read this, you may be getting worried. You have never heard of a study in second language research that collected data from a truly random sampling of people!
Does that mean that we cannot make any inferences from these studies to a wider population? Come to think of it, you may have noticed that most researchers do not identify the wider population that their sample comes from. If they tested English as a second language ESL learners at a certain university, is their sample representative of all ESL learners at that university, or all ESL learners in the entire country, or maybe the entire world? It seems to me that most people who write reports and most people who read them are content to let this issue be fairly vague.
Although not having random samples may compromise the external validity of a study, probably the best we can hope for in our field is that both readers and researchers will try to be careful in the limits of the generalizations they make and only draw those that are truly justified. If you are creating your own research project, you should consult a book such as Mackey and Gass , which gives more detail on research methodology as you decide how to sample your population What Does a P-Value Mean?
When you read a research report, you will have noticed lots of mind-numbing mathematical symbols in the Results section. If you are like most novices to statistics, you probably just skip over this entire section and go to the Discussion section, where the.
That can be OK in some cases but in other cases the author may not be completely accurate in interpreting the results or may overstate their case. If you think the only thing you need to glean from the results section is the p-value, then you are wrong. You won t be able to be a savvy consumer of research let alone carry out your own analyses unless you can understand more about all those symbols in the Results section of a paper.
So in this section I will try to introduce you to the reasoning behind a statistical test and what things get reported. At this point we have seen that the descriptive statistics about the sample which was tested should be reported, things like which first language background the participants have, and their mean scores on a test.
The next step the researcher will take is to perform a statistical test with the data, which means using inferential statistics. The main results that the researcher will report are the statistic, the degrees of freedom, and the p-value. Here is an example from Munro and Derwing , p. Munro and Derwing reported an F statistic of They additionally reported an effect size partial eta-squared of.
I will give a brief explanation here of what these numbers mean. First of all, even if the researcher does not explicitly state what kind of statistical test was used, you can tell what kind of test was used by the statistic that is reported. Each statistical test has a certain mathematical symbol associated with it. In general, the higher this number is and the greater it is than the number 1, the more likely the p-value is to be very small.
Munro and Derwing report a very large F-value of I will return to the question of what this statistic is presently. At this point, you can think of it as the result of a calculation involving mostly mean scores and standard deviations. I assume that you are familiar with what means or averages are, but I will elaborate on standard deviations in Chapter 3. For now, think of standard deviations as measuring how much people vary from the mean.
The next number you should see is something that represents the degrees of freedom. The degrees of freedom basically counts how many free components you have in. To understand what that means, let s imagine we have four contestants on a game show, and each gets to pick one of four doors, behind which there is either a great prize, such as a car or a boat, or a booby prize.
How many of the contestants have any freedom to choose a door? Figure 2. Contestant 1, Alice, can choose door 1, 2, 3, or 4. She chooses door 2. Contestant 2, Gary, can choose door 1, 3, or 4. He chooses door 4. Now Contestant 3, K Lynne, can choose either door 1 or door 3.
It s not as much choice as Alice had, but it s still a choice. She chooses door 3. Now the last contestant, Contestant 4, Eric, has no choice. He is left with door 1. In this case, we would say there were 3 degrees of freedom, because three of the four people have a choice, but the last person s choice is fixed.
We have only three choices that include some variation, and this means we have 3 degrees of freedom. When you perform a test that has group variation as well as individual variation, as shown in the quote by Munro and Derwing above, you will need to report two numbers for the degrees of freedom.
The first one deals with the amount of variation due to groups. Thus, even if Munro and Derwing had not told you, you would know that there were seven groups in the ANOVA test because the first number, for group degrees of freedom, is 6, and that means there were seven groups. You may wonder why you need to know what degrees of freedom are anyway. When researchers had to look up information for their statistical tests in tables, the degrees of freedom was a piece of information that was necessary to determine the critical value for finding statistical significance or not the number above which the statistical test would be considered significant.
Nowadays, if you use a computer to calculate your statistics which is what I am assuming you will do! However, you should still report it. For example, if you had a t-test, you would look up the alpha level you were interested in Table 2. As an example, if you calculated a t-value of 2. Another reason to understand what degrees of freedom are is that, in studies where authors may neglect to tell you the number of participants or groups they used, you can calculate this yourself by knowing the degrees of freedom.
And last of all, degrees of freedom can be a check on someone else s statistical work. If the degrees of freedom are impossible given what the author has told you about numbers of groups and sample size , you will know something strange has happened with the statistics. Finally in this section we want to understand what a p-value is. The p-value is the probability that we would find a statistic as large as the one we found if the null hypothesis were true.
In other words, the p-value represents the probability of the data given the hypothesis, which we could write as p d H o. Remember that our hypothesis is the null hypothesis, which says there is no relationship between variables or no difference between groups. So, in Munro and Derwing s results above, the p-value of less than.
This is a pretty small chance, and we decide we are safe in rejecting the null hypothesis and in assuming there are some differences among the groups. In general, in the field of second language research we use a cut-off point of 0.
In Chapter 4 I will also talk more about understanding p-values. But, for now, just memorize this phrase in order to understand p-values the correct way: Meaning of the p-value: The probability of finding a [insert statistic name here] this large or larger if the null hypothesis were true is [insert p-value].
In sum, the researcher will not just proclaim the p-value and declare their hypothesis to be statistically significant or not. They will in fact tell you, overtly or not, what kind of statistical test they used and report the value of that statistic, they will report the degrees of freedom associated with the test which can tell you something about the number of participants or groups if this was not mentioned , and they will report a p-value. I will argue in Chapter 4 that we can improve our statistics by not just reporting these three data points but also adding information about effect sizes, power, and using confidence intervals instead of p-values Understanding Statistical Reporting The previous section explained briefly how to interpret three numbers you will often see reported in experimental results the test statistic, the degrees of freedom, and the p-value.
Let s practice using results from real journal articles and see if you can begin to understand what those mysterious statistical reports mean. In this section I ll walk you through the answers; then I ll ask you in the next section to do it on your own. What does the r reported here refer to? The statistic r is used to give the results for a correlation. This report does not give the degrees of freedom. Why not?
Correlations don t report degrees of freedom; they report total sample size. This is quite likely, so we cannot reject the null hypothesis and. The second reported correlation has a p-value less than 0. What do you notice about the relative sizes of the r-values of the non-statistical versus the statistical results? The r of the first correlation is negative, but this doesn t make any difference. However, the second r-value is 0. A correlation can only range from 0 to 1, so you can see that relatively the 0.
What does the F reported here refer to? Using the degrees of freedom, calculate how many groups were tested. How many participants were in each group? The first degree of freedom 4 refers to the number of groups. There were 10 participants in each of the 5 groups.
This is a very low probability so we can reject the null hypothesis and accept the alternative hypothesis that the group categorization affected scores in perception and production. What does the t reported here refer to? The t-value reports the results of a t-test. Knowing that the degrees of freedom associated with a t-statistic equal the number of participants minus 2 1 for each of the groups , how many total participants were there?
By the way, it would be nice to know how much larger this number is than 0. What do you notice about the size of the t-statistic in this study compared to, say, the F statistic in question 2? The t-statistic is 0. The t-statistic is a lot smaller. Normally only one df is reported for this test, and that is the first number in parentheses given here after the statistic. The df is equal to the number of rows in the table R minus 1 multiplied by the number of columns in the table C minus 1.
There are two rows and two columns. This is a very small probability and we can reject the null hypothesis and conclude that there is a relationship between noticing questions and progressing to higher-level questions. What do you notice about the size of the statistic? It is 7, which is quite a bit larger than 2 and thus the p-value is quite low Application Activity: Understanding Statistical Reporting Now that you ve worked through the examples in the previous section, try a few more on your own: 5 French and O Brien , p.
How many people participated in this test? What do you notice about the size of the t-statistic? Polio, Fleck, and Leder did not report statistical results in words; they reported them in a table and then interpreted the inferences for their hypotheses. This part of the statistics tested whether there were differences in groups which received error correction or not. What do you notice about the size of the F?
Results revealed a significant relationship between formal instruction and type of verbal report [p. What do the rs reported here refer to? How many people s results were tested in this statistical test? The Inner Workings of Statistical Testing As a novice to statistics, when you read a research report it may seem that finding statistically significant results is the holy grail of research.
If a p-value of less than 0. It is much rarer to find in the published literature results that do not establish the hypothesis that the author was hoping for. Russell and Spada state that a publication bias against studies which do not find significant results for treatments is an established fact. When a study gets published where a treatment did not result in a positive result, the author will often try to explain the problem by referring to another explanation that could be plausible instead of just thinking that, especially in the case of studies with small sample sizes, the lack of an effect may be due to too little power; I argue this case in the last section of Chapter 4.
In this section I want to explain how hypothesis testing works so that it will seem less mysterious and you will be able to be more critical of the process. You already know from previous sections that the result of a statistical test is a statistic such as t or F, and there is an associated probability for that statistic. But how is that probability calculated?
Let s start from the beginning thinking about what a statistical test tests. To illustrate this process, let s take the question proposed by one of my students and listed earlier in the chapter. We want to know whether students of Spanish who play an online role-playing game will learn Spanish faster than students who meet with a conversation partner for the same amount of time every day.
So let s say we set it up so both groups do this extra work in Spanish for 30 minutes a day, three times a week. After a semester, we test our students using a item cloze test. We tested both groups at the beginning of the semester too, and we find that the game group on average gained 10 points on the test over the semester while the conversation group gained 7 points. There is thus a mean difference between the groups of 3 points.
Now there is always variability in measurement. For example, if the same Spanish student took the cloze test one day and then again a day later, that student might receive different test scores you might rightly argue that the increase was because the student had learned from the test, but let s suppose we had two different but equal versions and the student took a different one each time. The same kind of variability happens on a group level as well. The group who played online games might have had a gain score of 12 if they were tested on another day with a different but equivalent version of the test.
Since fluctuation is normal, we want to know what amount of fluctuation is not just due to chance alone. That is, if the groups differ by 3 points only, this. However, if the groups differ by 10 points, it will become unlikely that this much variation is due to simply chance.
As Crawley , p. That is why we need statistics. This part is where common sense can come in too. Statistically it may be possible to find a difference between our groups when there are only 3 points difference in gain scores, but you yourself should realize that a gain of 3 points on a point test is not very much.
Now this may be good news if students could use the role-playing game and get the benefits of talking to native speakers without having to actually seek out native speakers to talk to, then even if online game playing isn t spectacularly better than talking to native speakers it may be a much easier choice with pretty much the same results.
Do you see in this case that it really doesn t matter whether your statistics say the difference between the groups is significant or not? Even if the statistics say that the game-playing group is statistically better than the conversation group, the size of the difference is not huge this can be measured formally by something called an effect size, which I discuss in detail in Chapter 4.
And if the statistics say there is no difference between the groups, that doesn t matter, because playing an online game may logistically be easier than finding native speakers, but has good results. Now we return to how statistical testing works. In this case we would use a t-test to find out whether the difference between two mean scores is statistically significant.
The t-test is not mysterious it calculates a number based on subtracting the mean score of one group from that of the other group and then dividing this by a measure of the variation in scores that involves the standard deviation and the sample size. This calculation results in a number. This number is a point on the x-axis of a t-distribution. The t-distribution looks a lot like a normal distribution what you might think of as a bellshaped curve but is slightly different, and its shape has been calculated by previous statisticians depending on the degrees of freedom it has.
For example, the left panel of Figure 2. The two distributions are similar, but you can see that the normal distribution in black is more concentrated around the mean than the t-distribution in grey. The right panel of Figure 2. Let s say the value of the t-statistic is somewhere around 3. The probability that we would find a number this large or larger will be the area under the curve from the point of that t-statistic number to the end of the curve.
Actually, the area calculated will be from both sides of the curve, the extreme right and extreme left, if you are testing the hypothesis that the difference between groups could go either way; this is called two-tailed testing, and I will explain it further in Chapter 4. So Figure 2. Now you already know what the p-value means.
It is the probability of finding a statistic that large or larger but now you know why we say or larger, since the probability is counting the area under the curve for larger numbers as well if the null. On the left, the normal distribution the black line has less data in the tails the extreme ends than the t-distribution with 2 df.
If the probability of finding such a number is large, that means that it is quite likely we would find a variation this large in the sample just from random chance, and thus we could not reject the null hypothesis. Just because we cannot reject the null does not mean, however, that we must accept the null hypothesis in other words, we do not have to conclude that our treatment had no effect.
You might think of the analogy of a microscope, which needs sufficient power in order to see sufficient detail. In this case we might conclude that no conclusion could be made! Further testing with more participants would be necessary. Now in this example I demonstrated how the p-value would be calculated for a t-test using a t-distribution, but the other types of statistical tests also use their own distributions.
For example, there is a chi-square distribution and an F-distribution. Just like the 4 Some authors, including Wilkinson and the Task Force on Statistical Inference , believe that the phrase accept the null hypothesis is unfortunate and should never be used. Instead, we should say fail to reject the null hypothesis. Howell states that there is still much debate in this area and that some traditions encourage the idea of either rejecting or accepting the null hypothesis.
In order not to confuse novice readers further in this confusing area, I will use the term accept the null hypothesis instead of fail to reject. The area shaded under the curve is the probability that we will find a critical value this large or larger. The details of this don t matter to you, but what I do want you to understand is that there is no mystery to the entire process Application Activity: The Inner Workings of Statistical Testing 1 In your own words, explain what a statistic such as a t-value is.
Although what you normally read in a research report does, on the surface, all seem to hinge on whether that p-value is below 0. There is nothing magic about it, and if you have a p-value of this of course will not mean that what you have found is unimportant. Most authors will label a p-value of as approaching significance, but Kline points out that they.
In any case, as I will argue further in Chapter 4, too great a reliance on p-values obscures the more important issue of how important your results are, and how reliable they are in the sense of whether they could be replicated.
A confidence interval can provide the same information that a p-value does and more. In the example in Section I said the mean difference between the online game group and the conversation group was 3 points. This would mean that the difference between groups could be as high as 4. By looking at the confidence interval, we can see that there is no statistical difference between groups.
That is, because the mean difference may be zero since the confidence interval passes through zero , we know that the difference of 3 points between groups is not surprising. It most likely happened just because of chance variation in scores. But additionally we know something about how reliable and precise our statistic of 3 points difference is too. If the difference could be as large as 4. How can we say anything about future replications of the study? Well, here is where I need to clean up the precision of what I said in the previous paragraph.
Actually, that is not quite right. The confidence interval CI will give a range of values for a parameter such as a mean difference. What is the difference between a statistic and a parameter? When we perform a statistical test, whether descriptive or inferential, we label the number that we have calculated as a statistic.
Statistics are designated by letters such as X for the mean a sampling statistic or r for the measure of correlation between two variables a test statistic. Statistics are the measurements we take from our particular sample, but these statistics are really just guesses for the actual parameter that applied to the population.
What we really want to measure is the parameter of our population, and by basing statistical inferences on certain distributions such as the normal distribution or the t-distribution seen earlier in the chapter we can infer properties of our population parameters.
To summarize, confidence intervals provide the same information as p-values in the sense that they provide information about whether to accept or reject the null hypothesis, but they also provide additional data. If the confidence interval is very small we know that we have a precise estimate of the parameter, while if the interval is quite large we know that we are not very sure about the value of the parameter and that.
SPSS does not always provide confidence intervals for every statistic, but I will report them when possible throughout the book Summary of Hidden Assumptions Hopefully, after going through this section of the book you have realized that there is more to the results section to think about than just a p-value. First, you need to realize that the researcher has had to set up a null hypothesis in order to test their real and plausible hypothesis. Second, you should realize that, if the researcher uses inferential statistics such as a correlation or an ANOVA, they are hoping that the results of their research will apply to a wider population than just those sampled in the study.
Third, you have learned how to interpret what a statistic means, what degrees of freedom are, and how a p-value is calculated. Last, I have provided a little bit of information to show that p-values should not be the main way of deciding if research findings are important or interesting. Other types of statistics, such as confidence intervals or effect sizes discussed further in Chapter 4 , provide more information about such questions.
Another name for this kind of statistics is parametric statistics, and the reason for this should be clear to you now. Parametric statistics try to determine, from the statistics calculated from an actual sample, what the parameters of the population are. In other words, it uses what can actually be measured the statistics to estimate what the thing of interest that we can t actually measure is the parameter. Parametric statistics are based on assuming a certain type of distribution of the data, which is why there are many assumptions about data distribution that need to be met in order to legitimately use parametric statistics.
Parametric statistics are not the only kind of statistics available, however. Many people have heard about non-parametric statistics as well. Non-parametric statistics do not rely on the data having a normal distribution. Almost all of the parametric tests which are discussed in this book have non-parametric counterparts, and these tests will be addressed in Chapter Many researchers are reluctant to use non-parametric statistics because they have heard that they have less power than parametric statistics.
If we understand that the term power means the probability of finding a statistical difference when one exists, using either a parametric test or a non-parametric test when the data do not follow the assumptions can result in the loss of power to find statistical differences when they do in fact exist.
Thus it is not accurate to say that non-parametric tests always have less power than parametric ones. Non-parametric tests do not require that data be normally distributed, but they still impose assumptions on data distribution, such as the requirement that variances be equal across groups or across the data set the variance is a measure of how tightly or loosely bunched up scores are around the mean.
Maxwell and Delaney , p. Rank-based methods such as Kruskal Wallis can provide some protection against the effects of outliers data which is markedly different from the rest of the data , which parametric methods cannot. However, one cannot make a blanket statement about which kind of test is more powerful it all depends on the circumstances.
A mixed normal distribution or contaminated normal distribution looks much like a normal distribution in the middle part of the distribution, but has slightly heavier tails than a normal distribution. Wilcox laments that, although modern statistics has made great strides in improving methodology and practical applications for quantitative methods, the average researcher in an applied field generally has no knowledge of these modern advances.
Introductory statistics textbooks cover the classic statistics methods which were formulated prior to and, most tellingly, before modern computing power was available. In other words, classical procedures were not robust did not perform well in the face of some kinds of violations of assumptions More about Robust Statistics Advanced Topic Many textbooks will claim that parametric statistics are robust to violations of assumptions.
For example, you may have heard that the Student s t distribution the t-test is rather robust to violations of assumptions. According to Wilcox , the Student s t distribution upon which the t-test is based can perform accurately if both groups have identical distributions, equal variances, and equal sample sizes.
Student s t is not very robust to even slight departures from a normal distribution. Even just one outlier in a group can change our conclusion from rejecting the null hypothesis to one where we would have to accept it. This is because an outlier will inflate the group variance, and having a larger variance in the denominator will decrease the t-statistic.
The smaller the t-statistic is, the more likely we will reject. Wilcox , p. Suppose that we had two groups who received the scores shown in Table 2. The critical value for t is 2. We would say that Group 1 performed better than Group 2, because their mean score is higher, and the t-test tells us it is statistically different from that of Group 1.
Now consider what would happen if we increased the largest score in Group 1 from 13 to Clearly 23 would be an outlier to the data point in this distribution. It would seem then that we would have even more reason to find the groups to be different from one another. This example illustrates how even one outlier can have a large effect on the conclusions we draw when we use theoretical assumptions about our distributions.
It is true that most second language researchers, when faced with an obvious outlier like this, would probably just remove it, apparently removing the problem. I welcome you to check these details from my profile and jobs history on An ' Economist itself. Once agreed and the project has been started, I will submit routine progress for review. You will review and check if it is aligned to our procedure and requirements. I will continue to work and your feedback from the second step will be incorporated into the working phase to submit the modified and new stage.
Once completed, you will review the final output which will be already your wishful but still if you feel we might need to change it, I will incorporate the changes. It is because most of the times, once completed, projects are clearn enough to be visualized and hence final output format can be requested. I am sure the above plan when turned into specific of your project will help us both get this work in excellence.
Econometricians Club Attock, Pakistan. Led Econometrician at Econometricians. Club and responsible for producing high quality research products for clients in Academic and Industry. We specialization in Research Assistance to Academic and Industrial Research for business customers who need specialist solutions through Data Analytics.
I won the Dr. Hey experts, I am currently working on a project and I want to find out which channels of adjustments European banks have used in Hi, I need help on a Panel Data Analysis in econometrics using stata. I have a csv file with adj. We are looking for a Clinical Demand Forecasting Specialist.
Ideally, an experienced user of Optimizer capable of building and man Hi, I have the following SPSS problem: I have participants, all randomly assigned to one of three conditions leaving 40 part My study looks at students' vocabulary I am looking for expert in stata analysis of a data set Finance for my master thesis, running ols regression multivariate, univ We are looking for a spss expert to help us do data analysis of school level data, including HML modeling.
I need some data analysis done using eviews software for a project. We are a global market research firm seeking to talk with multiple experts in foreign exchange services - UK. Hi, I would requ I have data on insurance companies. I have done the basic data modifications but I am looking for someone able to help me with the Good Afternoon!
I need help with time-series analysis. I am repeating an existing research, but for a longer time-period. I have a Need some help on understanding code related to R programming and build some new code. This is based on recommender systems. We are looking for professionals with at least 2 years of current We have a project. I want to understand how to build better surveys to ensure I get the best out of respondents. I would like help in running a Multivariate Markov switching model.
Help needed urgently on Stata or eviews. I looked you up as I was drinking my morning Star Bucks Veranda Blonde, and contemplating the best, most convincing ar Looking for expert in Stata analysis of a data set, creating a do file and running OLS regression, limited time. Hello, hope this message finds you well. We require your SPSS expertise for a rather urgent matter.
My company was working on a da I have data from my car detailing business. I work out of commercial buildings detailing cars. There are three buildings that I Hi, I have an R programming assignment and I'm looking for an expert to help me solve 2 problems. Need someone to do the data analyzes for my honors thesis project.
One of the goals of the research project is to understand, ba We are a medical marijuana extraction start up looking to hire an independent contractor that can assist us with compiling and cre Hi, I have some questions that should be solved by stata software. If you can solve them and send me the answers with explanatio I need help doing a few questions on stata it's basically an econometrics and running regression and interpreting the data.
We are seeking well experienced and qualified Math Teacher for College level, Teacher can reside convenient country, teaching thro The system keeps crashing while running and I need a fresh pair of eyes I need a Stata expert with excellent command of user-defined maximum likelihood estimation methods. I need help with a probit model in stata. Need R programming tutor for predictive analytics classes Taking classes in Time Series and Marketing analytics and need someone We need help to make statistics in order to develop automated EMA setups vs.
Hi there, I have a data set that I need cleaned and analysed following standardized syntax. Any Stata analyst's looking for consul I am running growth regression on panel data 64 countries, 11 years to investigate the impact of Financial Development on econom I am seeking a statistician to conduct basic correlational statistics for my doctoral research, in the SF Bay area.
Looking for help with analysis in stata. Would prefer someone with good knowledge of econometrics to help me formulate which tests We are a prominent market research firm and are seeking a Data Analyst to join our talented and vibrant team. The Data Analyst mus I am looking for a statistician with high level of expertise in multinomial logistic regression analysis.
The selected person Please tell me regarding implementaion of GMM in dynamic panel model in stata and its explanation. Looking for a moving average exit strategy for ETF sector investments. I need an explanation for the time lag appearance of all mo Hi I am doing a Contingent valuation WTP study and struggle to select the appropriate tools and regression for the complex analy I need an Rstudio and Stata expert to help me with a project.
I'm working with a limited time frame, so please contact me as soon I have a panel data set. I tried to run OLS but normailty tests failed. Then i tried to run hetroscedastity consistent errors meth I have a limited knowledge with the program We are writing a research project on gas prices and how they fluctuate. In need of a research analyst!! I need help in a quantitative research analysis where I would need the help of spss anova etc to c Need a data on multiple regression for econometrics.
Need a case study on romer model for advanced country and its explanation. Hi Im looking for someone has expertise in both econometrics and eviews. I dont have specific project rather than im looking for The in Need to understand current market prices per hour rates for data analytics services in India by job role if possible by metro c I have a variable -total dollar value of transactions with a particular bank bank which is advising on the current transaction I'm also in a bit of a time-crunch.
I'd like a live tutorial and I can s I am really finding it hard to do Value at Risk. I am from energy industry and want to incorporate this onto the market prices and SPSS expert to run crosstabs and create professional tables for an extensive research project. Hi, I have entered all my research onto spss into two seperate data sets. I am really struggling on what tests to use and how to d As a Bay Area, California Santa Cruz County nonprofit research and education organization, we are moving into e-learning.
We are I am using stata to form portfolios for my research. I have daily data for all the stocks but I need help with how to form the I have a genomic data set which I would like to apply the R programming methodology too. In particular using the bioconductor pack Large company need a data analyst who is expert at using SPSS's data management capabilities not a statistical data analysis focu Hi, I am in need of some stata experts.
Im having problems with conditional prediction. Ive read through several manuals but still I am interested in running a panel data model to explain the push factor driver of the outward FDI from two Asian countries. I am looking for an individually who is preferably local to support me in SPSS research analysis. I am currently running relativel Specifically with interpreting data and creating plots. Will compensate for help if nec I have a health related project that needs editing and finessing.
Prefer US based stats person. I have a cross sectional data in stata version. I need help with merging the individual and household datasets. Referring to the d Contracts are in place with a number of diffe I need to open a stata data, I'm working with rstudio. I need someone to explain how I can do this, or at least how I can convert I need to plot the adjusted mean or geometric mean ,in my example' of systolic blood pressure as a function of age, sex and overwe I am looking for some assistance estimating a demand system using GMM in repeated cross sectional data.
The data set is large and Hi, I want to analyse day of the week and end of the month effects on the stock market. I have daily and monthly closing prices of I need an economist who has worked with brain injured plaintiffs before to assess the loss of earning capacity for a client. Looking for Subject Matter Expert SME who can help us create training material and help do so training related data analysis, in Looking for expert in Stata analysis of a data set, creating a do file, selecting regression models.
I'm looking for a person that is an expert with a statistical software called SAS and can do a Conjoint Analysis using this softwa Can you help me with an econometrics assignement stata? I am running out of timeWe can discuss a price. I need to build a Poisson model in Stata.
It doesn't have to be big or complicated, but I have very limited Stata experience. I am running a regression on panel data to investigate the determinants of FDI. Its my first time using stata. I need tutoring thr Need to find a STATA data set for the topic of development economics ideally for the effect of political institutions, i can provi When I run my regressions and want to use outreg2 I receive an error.
Can anyone tell me how to solve this? When I use the outreg How to build a simulation model in excel without use of add-ins during health economic evaluations. I need help in running some regressions in Stata. I have limited knowledge on the subject. I have a large data set of conflict states from and a number of variables such as GDP levels unemployment levels etc.
I have completed and rolled up my doctoral quantitative research data. Have interview notes and many research papers I want to cor I am doing a research proposal for my dissertation. My research design is a one group pre test post test design evaluating the imp Hi, We need a statistician who can analyze the survey questionnaire and prepare the list of insights that can be derived using ad
|Gestion environnementale profitable investing||Reinvesting dividends tax rate 2014|
|Binary options trading brokers||887|
|Unianova in stata forex||Date of ferrari ipo|
|Investing for retirement uk to rsa||36|
|Forex trading strategy free download||Plan reorganizare servtrans investing|
|Big forex players||301|
|World forex clock||What group will participants be in will there be a control group that does not try to learn any vocabulary at all? I assume that you are familiar with what means or averages are, but I will elaborate on standard deviations in Chapter 3. The remaining nine chapters of Part II go over various types of statistical analyses that are common in the field of second language research. Looking for Subject Matter Expert SME who can help us create training material and help do so training related data analysis, in This is what you want, so go ahead and press OK! I thank the reviewers of the book as well for their careful work in reading the manuscript and providing useful feedback. Would prefer someone with good knowledge of link to help me formulate which tests|
|Unianova in stata forex||But usually a research report will not stop with descriptive statistics. In this example I took a continuous variable AGEone that was not a group already, and collapsed the numbers into groups. After I finish unianova in stata forex the condition I want to keep data in, I have a choice in the Output section of Figure 1. I have a variable -total dollar value of transactions with a particular bank bank which is advising on the current transaction Spss Modeling Project. That is, if the groups differ by 3 points only, this.|
Blocked the partner steady, the default easily demonstrate this in best. Click is plans more. View it support.
Compute and interpret the simple main effects of teacher across textbook: In SPSS, the data of Table appears as follows: UNIANOVA. I have submitted a paper on Journal of Stock and Forex Trading of Longdom I´ve mostly used Stata earlier on, but as the stock options and the underlying. The primary end-point was evaluated by factors using UNIANOVA, and post-hoc tests were done. A multiple regression analysis was carried out to identify.