#09 SYSTEMATIC OBSERVATION & CONTENT ANALYSIS

Key Takeaways

[Music] hello and a warm welcome I am armed interest professor for organizational behavior at the foot baingan University in Germany and this is my course on social research methods welcome back meanwhile we were talking about two ways how you might collect data as part of Social Research one was testing and the other one was simply asking questions by maybe using questionnaires running interviews and the like and now I would like to share with you two more methods of data collection the one is the systematic observation and the other one is the content analysis so let's start with systematic observation so sometimes you want to observe people this is what we do in social research very often we observe human beings we don't observe animals we do not observe clouds we do not observe plants stones we we observe people human behavior and that's pretty complex and I will show you what the challenges will be when doing so maybe I can start with a simple example another simple one but a very famous example in 1971 this great Philip Zimbardo conducted an experiment that became known as the Stanford Prison Experiment so while semester break a couple of Maine students took part in an experiment and then they were randomly assigned into two roles one worse that were the guards and one were the prisoners and the idea was that they they have reshaped the faculty facilities into a prison so early so there were really Brittain cells and all alike so there the role of the guards was to put it simply it was a little bit more complex a very readable book by the way you should really read it the Lucifer effect it's or you can also watch this movie the standard Stanford Prison Experiment it's it's every psychologist should see this movie it's it's excellent really it's pretty close to to what's in bottle as described in his own book so the role of the guards were to make sure that the prisoners follow some rules that was it that was the only task one rule for instance was that the people should not call each other by their names but just by their numbers they had numbers tacked on their prison like cells that was one rule and I don't want to go too deep into this experiment the question was what what what what is the effect of taking over roles what is the effect of having power over other people what happens then things like deindividuation and the like was also examined so a lot of things psychologically spoken extremely interesting experiment so the interesting thing here in our regard was that these these scars especially the guards not so much the prisoners but the guards were constantly observed there were cameras of course but they were also one-way or two-way mirrors so you know these mirrors were from one side they are served like windows from the other side they serve as a mirror so Philip Zimbardo and his colleagues were capable to to constantly observe the God's behavior and this is a typical example for observation you observe people and now I will come back to this example but whenever we talk about observation or to be more precise about systematic observation there are different types of observation and that's first interesting thing here sometimes observation is open and sometimes observation is hidden what's the difference when observation is open the people who are observed know that they are observed okay that's the only thing and that has the danger of reactivity we were talking about this when people know that they are observe they might not show their natural behavior because they for multiple reasons we don't have to go too deep into this okay when people are do not know that they are currently observed that they actually take part in a study they might rather show their natural behavior and this is what we want brain so open hidden the sometimes observation is participating that means that the researcher they interact with the subject they are part of them so sometimes we have this sometimes when you do research about the culture of let's say a company or a try you you you become part of it you you you you shadow these people you spend time with them you are a part of them open or hidden that that's not the point here it's not the question of whether the people know that you observe them or not that was the first dimension this dimension just about do you actively interact with the people or don't you and if you if you do not participate like like in the Stanford Prison Experiment Philip Zimbardo did not and now I must be careful yeah he is dude during the day he was not participated by depending but at the end he was participating in Christie Christie got confused Philip Zimbardo played a role in that game and the funny thing is that's why a struggling is in the we found out and it is writes about this in his book he's excellent he found that he played a major role in this entire thing because he was the so to speak that the director of the prison he had much power over the gods so he was seen like this so to some extent he was participating but in the daily behavior of the gods he was not participating he was not there he did not spend time in that prison yeah so I would say to large extent he was not participating even though Philip Zimbardo interacted with the gods at some point of times of course this is one another thing right participating non participating so when you when you write about your study and you have done a kind of systematic observation would be good if you named it that way was it a participating among participating hidden or open and here's a third differentiation sometimes participation observations or a participation happens in the field what is that field when you when you use the term field in social research we do not think of a field in nature like an acre or so when we when we use the term field in social research we mainly think of the real world right oh you couldn't put it that way you do the observation in a setting that that would exist even though you would not do the study okay so for instance when you observe people on the street and you can observe a lot when you watch people on the street that's the field when you observe children in the classroom it's the field okay the the field is laboratory and the Philip Zimbardo experiment the Stanford Prison Experiment was an observation in a laboratory because the whole setting was created just for the purpose of this study and this setting would not exist when the study would not exist so that was a laboratory so field laboratory ok so so we better are clear on what we actually are doing right so now here's the more important thing when you observe people then it's a case where a human being is observing another human being one monkey is observing another monkey so to speak no offense please but is it possible that a human being can observe another human being in a very objective way is that possible at least it's extremely possible extremely difficult and why is that to to to make this a little bit more clearer there was a test that was invented very early in the in the in the 20th century it was actually it was in the forties by moray this test was called the the Thematic Apperception tests ta we also only say TI t in psychology and what is that it's a projective test what does that mean projective whenever you see something what you see depends on the viewer and not so much on what you see so different people see different things in different things when different people observe different people or one in the same person let's put it that way different people observe one in the same person they will see different things and why is that it's because people are different people have different ways of looking at things so Marie has developed a test based on exactly this idea so what he did is he he showed the subjects different pictures just pictures drawn pictures maybe on one picture there is uh is a couple a man and a wife and it's a kind of I would I would say but maybe you don't see that way you would not see that way that's a very emotional situation in another picture there is a train leaving a station and somebody is crying I think it's crying in another scene there is a rapid on an acre on another scene there is one women watching another women from a hidden position in another picture there is an older man and a younger man having some kind of debate so in your task in this test and that's amazing I did it myself really I learned a lot about myself by the way I don't share what is different people and in their task is that you have to tell a story based on the picture that's the idea you have to tell a story based on the picture you see the picture of this women watching another women from a hidden perspective and now you have to tell a picture you see a man having a debate older man having a debate with a young woman tell tell a story what do you see in this picture and when you when different people tell different stories they do it because they see different things in one in same picture and why do they see different things because they differ and and that's exactly what we named subjectivity that's subjective seeing different what you see what one observes depends on the observer and not so much on the person you observe that's the original meaning of being subjective and this is what we don't want right so what we want instead in social research method is that when different researchers see the behavior of different people they come always to the same conclusion then its objective this is what we want okay and what what are what are the challenges now mainly one challenge is that when you when you observe another person you cannot not interpret what does that mean you see another person saying something you see another person showing some body language and we cannot we cannot just see the body we we always see the language as well okay it's very hard for us to just see the behavior when we see another person and there are tears enough in their face we we assume this person is sad and we cannot just see our tears in the face move interesting interesting tears sad of course you see if you cannot not you cannot not interpret this as sadness yeah you cannot and very often we see what we expect I mean that's a that's a fundamental psychological idea and then those who a little bit deeper in psychology you know this you you you only see what you know you only see what you what you expect anyway so and we always expect behavior from others right we always see this so so I mean for instance when it comes to social judgment I mean that that's something that really applies in that matter meaning one person is looking at another person and make sense of it so if you have a certain stereotype of specific people right could even be racist 'ok so if you think that white people white old man behave like this and you expect this because this is your stereotype but old white man you gonna see it you're gonna see what you expect and you will interpret behavior so that the so that your interpretation it goes along with your prejudice with your stereotype with the social scheme that you you you you you have if there is a punk yeah walking across the street and you think punks are lazy whatever you see with this Punk is the outcome will always be look again here is it again lazy is it you see it because it expected but maybe this Punk is not lazy punk is working actually but you don't see it it can have thousands of examples you have to be absolutely clear and I will come to the practical implications in a minute we have to be absolutely clear about the behavioral behaviors that you want to observe and also classify so when Philip Zimbardo we watching the gods in this artificial prison he had some kind of schemes he knew exactly what he what what kind of behavioral patterns they want to want to track or want to document or what to record so for instance there is a prisoner resisting the order of a God now as Philip sabado knew there are maybe three or four different behavioral patterns that a guard might show and he knew this little bit in advance but then he looked does the guard shall behavior a B C or D right so having having this kind of structure is extremely important and integral we'll come to this very often a challenge is in practice that when you observe people in the field that the behavior you want to observe happens in a very short period of time so for instance you want to observe what should I take you want to observe children on the on the schoolyard and you want to observe the interaction whatever you want to observe yeah that can can happen that that happens very quick in a very particular moment and one second later the behavior is already gone sorry sometimes we have to be very very quick in your observation you're very very fast and still your observation is supposed to be precise so better be prepared okay it's always good to have more than one observer and because once you have more observers you can increase reliability because the individual arrow the individual bias may be mice might be zeroed out so having more more observer than one and then you really have to calculate and I will come to this point later you have to calculate the interrater reliability or what we also name objectivity and we're talking about this in a previous session when we talk about validity objectivity and reliability you really have to calculate this different researchers really must come to the same conclusion when observing in depend otherwise the way they observe is not reliable and something that is not reliable cannot be valid and if something is not valid go home forget about your study okay there is another thing that is pretty similar to to what I have explained with the pretest when we were talking about questionnaires human behavior is so so diverse so so full of variety that in reality you will find much more behavioral patterns ways of behavior that you would ever expect in advance so when you when you observe children in the schoolyard for instance when you want to observe consumers on a in a store when you want to observe whatever as I said it's it's very important that you be clear about what you want to observe otherwise it will be overwhelmed you have to have a clear structure about what you want to observe but in the end when you go into the field especially and you see the behavior of people you will find much more behavioral patterns than you ever have expected in advance so be prepared about this variety of things the consequence of this and I will summarize this in the end of this episode is that always test what you gonna do always go out you want to observe people in the field okay go out and do it be prepared and check and test whether your way of observing things really works well okay so another thing is sometimes when you have a participating observation a participating observation means that you interact with the subject then there might be kind of reactivity the subject reacts on you you cannot not affect others when you interact with them you cannot not really not you cannot right so yeah and especially it's it's difficult when the subject knows that they take part in the study so it's a when its open participating observation that's that's very difficult because then you really have to make sure that you do not affect the subjects in a way so that it supports your hypothesis if you expect a certain behavior on the other side of the subjects and you interact with them you might even on an unconscious level in a subliminal level you might influence them and at the end you get the results you expected but that's not a good way of doing things okay so this can go on and on and on what I want to tell you here is that systematic observation is damn difficult really it's really really difficult when you do something like this I like it I found it absolutely call and an observation very often is the better way of collecting data than always just sharing questionnaires oh we already have too many questionnaires sometimes I feel so so observation is absolutely cool but very very very very very difficult okay so here is another method of data collection which is the content analysis and what is that in this case content analysis you do not observe people but you use existing material to in prett behavior of people okay so what does that mean maybe I give you one or two examples a nice example I found recently was a study that was conducted in Germany about how kids see their future okay so you cannot you cannot share a questionnaire with kids that's that's difficult I they don't have fun with reacting to questionnaire maybe you could but it's difficult it's not a very I would I would say a questionnaire is not the most appropriate method to collect data about behavior and experience of kids it's too boring too adult ish I don't know how to say this so the researchers they took another route so they asked the children to write an essay a short essay and the title of the essay was my future can you imagine this children writing an essay one page about my future then you take this 100 essays so now you have content yeah this is not observation this is not questionnaire this is not testing now you have content and you analyze the content and this is what we name content analysis but now you have this stack of essays how do you analyze this how do you do this right again it's very very difficult you can you can look at some terms that appear you can look at some typical structures some typical ideas that that that show up in this that you find in these essays things like this you know I worked a lot in human resource management in my career and I very often heard the from CEOs saying people are our most important asset you also find this sentence very often on the websites of many companies people are our most important assets bla bla bla bla bla yeah so okay I did not believe this that it's really true so we know that those countries those companies put on the capital market they have to publish an annual report so what we did in a project team was we collected the all the annual reports of the biggest companies in German it's like a fortune 500 okay so we had all the annual reports that was a huge stack of reports and we analyzed all these reports and the question was what is the relevance of people in the annual reports when when people are the most important asset really then we would expect that the people the employees yeah that they somehow appear in the annual report what do we do with the employees what are their career opportunities what is our health management what is our training activities what is I don't know what what is the company's strategy to hire new people what is the unborn strategy what is how its employee satisfaction in this organization how does the company make sure that the leadership quality is appropriate all those things so we come we analyze the content of this annual reports so again this is not questionnaire this is not observation this is not testing this is analyzing available content okay also you can analyze the career websites of many companies looking okay how do they present themselves you can analyze you can analyze advertisements of various companies you can analyze commercials regarding different things recently came across a study about gender discrimination in in commercials yeah so one student had watch and analyze these commerce on a very very very structure systematically so again this content analysis yeah looking at these commercials looking at the essays looking at the career websites looking at what did I mention all these things yeah you do not just read and then think hmm what it was there no you really have a very structured approach in analyzing all those things that's essential yeah that's content analysis really important and you can do a lot of wonderful things based on this because also in times of the internet there are so many is so many information available when you can do content analysis based on the on the Facebook presence of people can you imagine us how many create studies you could do just with the information you get on social media I mean compare different Instagram accounts you can compare you can analyze all sorts of things and again it's not questionnaire it's not it's not observation it's an analyzing content that exists okay so observation systematic observation content analysis so let me finish with some general recommendation that apply for both okay and this is goes pretty much along with what I already have set with developing a questionnaire yeah be clear about what you want to observe that's the first thing okay it's exactly the same with when when you do questionnaire be clear about the variable you want to measure be clear about the theoretical concept what's in what's out what is it what you want to observe what is it what you analyze and what what not and then menu when you analyze behavioral patterns yeah be clear in advance what are these behavioral patterns that you want to observe yeah really you have to anticipate in that regard right or when you content analysis what is it what you specifically want to look at are these specific sentences are these specific words are these specific ideas are there specific lines of argumentation and what is it yeah be very very specific on what you want to observe and this can be extremely complex so one recommendation I can give you here is you sometimes can also work with prototypes prototype so prototypes are typical cases typical behaviors it's more than just the individual single behavioral pattern this is a cluster of behavioral patterns so for instance when you you want to you want to observe children on a schoolyard and you want to see the level of interaction right so because you assume okay there are some people who do not interact so much and there are other kids who interact like hell you know so you want to analyze this maybe your question is is there a relationship between social status and social interaction maybe there is one so now when you observe social interaction of kids that's extremely complex I mean you you can't imagine what happened with kids they it's crazy they have so many behaviors but now you still want to say okay I want to observe social interaction what do you concretely observe have some criteria have some things and now based on those criteria that you might might anticipate and describe in advance you might be able to prototype different behavior saying okay what we see is there are four levels of social interaction this is the first level of interaction is the silent guy so these are typically children who just stand alone don't talk don't play with others and they do this over a long period of time so this is prototype number one prototype number five now let's go to the other extreme our children's to constantly talk to other people they always talk and they they talk and when they talk to another they very much dominate the other person so it says are very very active and never stand alone always talking and dominating other people and also changing different people they so this is the other extreme so you you describe this prototype based on various a criteria so now you have prototype one two five and you have some some between now when you observe kids sometimes that don't make your life is you just look at okay is that prototype one two three four five that helps to reduce the complexity but be careful really this criteria must be clear in advance but prototypes very often help the same aspire the way with with content analysis so that you summarize things a little bit especially with observation when you sometimes have to to to make a judgement in a few seconds it's very often easier to have this kind of prototypes you must have something and that's the important term a coding system or a coding scheme and that's the equivalent to the to the questionnaire right a coding scheme coding system is something that you have available while you observe the people this is where you make notes where you have the behavioral patterns ready where you instantly can track the behavior of individual people right you have to have this scheme in this tool and once you start doing observation systematic observation or content analysis as I already mentioned make sure that you do it in groups of two right and then start doing things individually first so you have two observers or two analysts doing a Content analysis and they do things independently so maybe when I go back to this example of the annual reports okay have two researchers two analysts who independently analyzed let's say 10 15 reports first and then you compare the results and hopefully these results correlate and this correlation tells you how the interrater reliability is so if you forgot about interrater reliability and so go back to the episode about validity reliability and objectivity itis I just explained it there so really it did it so so so essentially also in my content analysis that after I always have this kind of check point where we have many researchers many analysts and they observe things and then we have a check and say okay now let's check interrater reliability is that is that okay and it should be at least point seven or 0.8 if you don't achieve a level of point seven point eight inter alia rau interrater reliability on observational content analysis you should not start the study really not make sure that that you are there already if you aren't there already it's very important that you either optimize your coding seam scheme your coding system because maybe that's still not precise enough or you train the observers or the analysts better one of the two applies here right so okay so do this and when you when you have done an observation especially in a in an open observation when you really have observed people in an open way either in the field or in the laboratory laboratory very often it's open I mean why should a subject be in a laboratory without knowing that he or she will be observed I mean that's why they are there but sometimes if you have this one one-way mirror people do not know that they are observed but I mean they can expect this okay but now you have subjects in a laboratory that's a open open observation what you very often can do is to make a debrief and that's that's really reasonable I mean think of again Philip Zimbardo what he did was after after the experiment was over and he had stop this experiment much earlier than it was planned because really I mean that turn out to be such a dangerous experience especially for the prisoners really they they really got worried that some prisoners got killed early it was a all run crazy it was a little it's a brutal experiment so he had to stop this experiment and then he made a debrief with the guards and the interesting thing it's a kind of interview right and in that case there was a group interview so he had all the guards sitting around the table and then he asked a difference God hey God we observe we observed that you have behaved this way or this way why do you think have you behave in that way and then you gain some additional information that might help you to interpret the behavior of your subject subject or when we think of the Milgram experiment you know the famous muqam experiment which is you know you probably remember this it's the experiment where you have a learner and a teacher and the learner is supposed to learn and when the learner does a mistake he will receive a punishment by the teacher through an electric shock and the electroshock was fake and this experiment was about the influence of authority on people right so the experimenter was the authority telling the teacher you have to punish this person and then they just looked how far they went of course all the teachers who were the real subjects in that experiment they were observed again this was a this was a observation in the laboratory so and you see all kinds of different behaviors for instance there was one subject who was laughing really I mean this this this this subject that he punished another man and he here the other man crying you know I mean that was an actor it's not really crying was from a tape but the teacher must must must must assume that he really punishes another person and the subject was laughing 200was and then dr. Milgram was asking why were you laughing why were you laughing because that's a very strange unexpected behavioral pattern that you found here and debrief can be very helpful so okay so systematic observation content analysis that's that's for the moment and I think that were all the the methods that we use in social research there might be some more about but these are the most important ones yeah testing asking questions observing content analysis so that's for the moment thanks for listening thanks for watching see you next time [Music]