disadvantages of using longitudinal study in research

17 Longitudinal Study Advantages and Disadvantages

Longitudinal studies are a research design which requires repeated observations of the same variables over specific time periods. These may be shorter examinations or designed to collect long-term data. Under most situations, it is treated as a type of observational study, although there are times when researchers can structure them as more of a randomized experiment.

Most longitudinal studies are used in either clinical psychology or social-personality observations. They are useful when observing the rapid fluctuations of emotion, thoughts, or behaviors between two specific baseline points. Some researchers use them to study life events, compare generational behaviors, or review developmental trends across individual lifetimes.

When they are observational, then longitudinal studies are able to observe the world without manipulating it in any way. That means they may have less power to detect casual relationships that may form in their observed subjects. Because there are repeated observations performed at the individual level with this option, there is also more power than other studies to remove time-invariant differences while review the temporal order of events that occur.

The longest-running longitudinal study in the world today was started in 1921 by psychologist Lewis Terman. He wanted to investigate how highly intelligent children would develop as they turned into adults. The original study had over 1,000 participants, but that figure has dropped to under 200. Researchers plan to continue their work until there are no participants left.

These are the crucial longitudinal studies pros and cons to review before setting up this form of a panel study.

List of the Pros of Longitudinal Studies

1. This form of research is designed to be more flexible than other options. There are times when a longitudinal study will look at one specific data point only when researchers begin observing their subjects. You will also find that this option provides enough data when implemented to provide information on unanticipated relationships or patterns that may be meaningful in specific environments. Since most of these studies are not designed to be lengthy, there are more options to pursue tangents here than in other research formats.

Researchers have an opportunity to pursue additional data points which were collected to determine if a shift in focus is necessary to review a complete set of information. If there is something interesting found in the material, then longitudinal studies allow for an option to pursue them.

2. The accuracy rate of the data collected during longitudinal studies is high. When researchers decide to follow longitudinal studies to collect observational data, then the accurate rate of the information they collect is high because everything occurs in a real-time situation. Although mistakes do happen because no one is perfect, the structure and foundation of this option limits the problems that can occur. This information is also useful in the implementation of changes that may be necessary to achieve the best possible outcome during an observational period.’

3. This research method can identify unique developmental trends. When researchers pursue a short-term longitudinal study, then they are looking for answers to very specific questions. If a long-term model is developed, there is an opportunity to identify specific developmental trends that occur in various fields, including sociology, psychology, and general medicine.

Researchers using longitudinal studies have opportunities to track multiple generations in specific family groups while still collecting real-time data on all of the individuals being tracked to see how current decisions can influence future outcomes for some population demographics.

4. It allows for the consistent use of the observational method. It is a simpler process to collect information when using longitudinal studies for research because it almost always uses the observational method. This structure makes it possible to collect consistent data samples at the individual level instead of relying on extrapolation or other methods of personal identification. It is the consistency offered in this approach which provides for exclusion differences for individuals, making it possible to exclude variations that could adversely impact outcomes as it happens with other research options.

5. Longitudinal studies allow for unique a specific data points to be collected. Most research study options provide a structure where data is available over a short time period for collection, offering a small window where cause-and-effect examples can be observed. Longitudinal studies provide an option to increase the amount of time provided for researchers to collect their data, sometimes on a very dramatic scale. There are some studies which are measured in decades or centuries instead of days, weeks, or months. This process makes it possible to examine the macro- and micro-changes that can occur in the various fields of humanity.

6. This process allows for higher levels of research validity. For any research project to be successful, there are laws, regulations, and rules that must be instituted from the very beginning to ensure all researchers follow the same path of data collection. This structure makes it possible of multiple people to collect similar information from unique individuals because everyone is following the same set of processes. It creates a result that offers higher levels of validity because it is a simpler process to verify the data that is being developed from the direct observations of others.

7. There are three different types of longitudinal studies available for use. Researchers have access to three significant types of longitudinal studies to collect the information that they require. Panel studies are the first option, and they involve a sampling of a cross-section of individuals. Cohort studies are the second type, which involves the selection of a group based on specific events, such as their historical experience, household location, or place of birth.

The final option is called a retrospective study. This option looks at the past by reviewing historical information, such as medical records, to determine if there is a pattern in the data that is useful.

List of the Cons of Longitudinal Studies

1. The structure makes it possible for one person to change everything. Longitudinal studies have a robust reliance on the individual interpretations that researchers develop after making their observations. That makes it possible for personal bias, inexperience, or a mistake to inadvertently alter the data being collected in real-time situations. This issue makes it possible for the information to be invalid without researchers realizing that this disadvantage is present in their work. Even if there are numerous people involved with a project, it is possible for a single person to disrupt potentially decades of work because of their incorrect (and possibly inadvertent) approach.

2. It is more expensive to perform longitudinal studies than other research methods. This disadvantage typically applies to the research studies which are designed to take longer periods of time to collect relevant information. Because observations may last for several years (if not decades), the organizations which are behind the effort of information retention can discover that their costs can be up to 50% higher in some situations when they choose this method over the other options that are available. Although the value of the research remains high, some may find the cost to be a significant barrier to cross.

3. The information collected by researchers may have few controls. The real-time observational data that researchers collect during longitudinal studies is both informative and efficient from a cost perspective when looking at short-term situations. One of the problems that this method encounters is that the information being collected comes from a relatively small number of individuals. Unless it is built into the rules for collection, there may be no controls in place for environmental factors or cultural differences between the individuals involved.

4. It can be challenging for longitudinal research to adapt to changes. There is sometimes no follow up to identify changes in thinking or operations that occur when using longitudinal studies as the primary basis of information collection. Researchers sometimes fail to compare attitudes, behaviors, or perceptions from one point of time to another. Most people change as time passes because they have more information available to them upon which they can draw an opinion. Some people can be very different today than they were 10 years ago. Unless the structures are flexible enough to recognize and adapt to this situation, then the data they gather may not be as useful as it should be.

5. Longitudinal studies often require a larger sample size. Researchers use longitudinal studies to develop a recognition for patterns and relationships. That means there is a large amount of data that must be collected from numerous individual sources to draw meaningful connections to the topic under study. If there is not a significant sample size available to researchers for the project, then there may not be enough information available to find specific conclusions.

Even when there is enough data present for researchers to use, the sheer size of what they collect can require data mining efforts that can take time to sort out.

6. Some people do not authentically participate in longitudinal studies. As with any other form of research that is performed today, you will encounter individuals who behave artificially because they know they are part of a longitudinal study program. When this issue occurs, then it becomes challenging for researchers to sort out what the authentic and inauthentic emotions, thoughts, and behaviors are from each other. Some participants may try to behave in the ways that they believe the researchers want to create specific results.

A study by psychologist Robert S. Feldmen and conducted by the University of Massachusetts found that 60% of people lie at least once during a 10-minute conversation. The average person will lie 2-3 times during that discussion. The content of fibs varies between men and women, trying to make themselves look better or to make the person they are talking to feel good respectively. Researchers must recognize this trait early to remove this potential disadvantage.

7. Longitudinal studies rely on the skill set of the researchers. The data that longitudinal studies collects is presented in real-time to researchers, which means it relies on their individual skills to make it useful. Those who are tasked with this job must follow a specific set of steps to ensure that there is authenticity and value to what they observe. Even if you offer step-by-step guidelines on how to perform the work, two different researchers may interpret the instructions differently, which can then lead to an adverse result. The personal views of the information being collected can also impact the results in ways that are not useful.

8. The data that is collected from longitudinal studies may not be reliable. Although the goal of longitudinal studies is to identify patterns, inaccuracies in the information collected can lead to incorrect interpretations of choices, thoughts, and behaviors. All it takes is one piece of data to be inaccurate for the results to be impacted in negative ways. It is possible that the findings of the research could be invalidated by just one incorrect interpretation of a real-time result. That is why any conclusion made using this method is often taken with a “grain of salt” with regard to its viability.

9. There is a time element to consider with longitudinal studies. Researchers may find that it requires several years of direct observation before any meaningful data becomes available through longitudinal studies. Some relationships or observable behaviors may never occur even though it seems like they should, which means this time investment may never offer dividends. These studies must have the means to maintain continuously open lines of communication with all of the involved parties to ensure that the quality of the data remains high throughout the entire effort.

10. Longitudinal studies always offer a factor of unpredictability. Because the structure of longitudinal studies will follow the same individuals over an extended time period, what happens to each person outside of the scope of the research can have a direct impact on the eventual findings that researchers develop. Some people may choose to stop participating in the study altogether, which may reduce the validity of the final result when published. It is possible for some individuals or households to shift their demographic profile so that they are no longer viable candidates for the research. Unless these factors are included in the initial structure of the project, then the findings that are developed from the work could be invalid.

The pros and cons of longitudinal studies provides us with a valuable foundation of data that makes it possible to recognize long-term relationships, determine their value, and where it may be possible to make healthy changes in numerous fields. There are certain risks to consider with this process that may create unpredictable outcomes, but it is also through this research method that we will likely find new ways to transform numerous scientific and medical fields in the future.

13 Advantages of Disadvantages of Longitudinal Studies

Longitudinal studies are a method of observational research. In this type of study, data is gathered from the same subjects repeatedly over a defined period. Because of this structure, it is possible for a longitudinal study to last for several years or even several decades. This form of research is common in the areas of sociology, psychology, and medicine.

The primary advantage of using this form of research is that it can help find patterns that may occur over long periods, but would not be observed over short periods. Changes can be tracked so that cause and effect relationships can be discovered.

The primary disadvantage of using longitudinal studies for research is that long-term research increases the chances of unpredictable outcomes. If the same people cannot be found for a study update, then the research ceases.

Here are some additional key advantages and disadvantages of longitudinal studies to think about.

What Are the Advantages of Longitudinal Studies?

1. It allows for high levels of validity. For a long-term study to be successful, there must be rules and regulations in place at the beginning that dictate the path that researchers must follow. The end goal of the research must be defined at the beginning of the process as well, with outlined steps in place that verify the authenticity of the data being collected. This means high levels of data validity are often available through longitudinal studies.

2. The data collected is unique. Most research studies will collect short-term data to determine the cause-and-effect of what is being researched. Longitudinal studies follow the same principles, but extend the timeframe for data collection on a dramatic scale. Long-term relationships cannot be discovered in short-term research, but short-term relationships can be tracked in long-term research.

3. Most will use the observational method. Because longitudinal studies will use the observational method for data collection more often than not, it is easier to collect consistent data at a personal level. This consistency allows for differences to be excluded on a personal level, making it easier to exclude variations that could affect data outcomes in other research methods.

4. It makes it possible to identify developmental trends. Whether in medicine, psychology, or sociology, the long-term design of a longitudinal study makes it possible to find trends and relationships within the data collected. It isn’t just the span of a human life that can be tracked with this type of research. Multiple generations can have real-time data collected and analyzed to find trends. Observational changes can also be made from past data so it can be applied to future outcomes.

5. Data collection accuracy is almost always high. Because data is collected in real-time using observational data, the collection process is almost always accurate. Humans are fallible beings, so mistakes are always possible, but the structure of this research format limits those mistakes. That data can also be used to implement necessary changes that a course of action may need to take so the best possible outcome can be identified.

6. Longitudinal studies can be designed for flexibility. Although a longitudinal study may be created to study one specific data point, the collected data may show unanticipated patterns or relationships that may be meaningful. Because this is a long-term study, there is a flexibility available to researchers that is not available in other research formats. Additional data points can be collected to study the unanticipated findings, allowing for shifts in focus to occur whenever something interesting is found.

What Are the Disadvantages of Longitudinal Studies?

1. There is a factor of unpredictability always present. Because longitudinal studies involve the same subjects over a long period, what happens to them outside of the data collection moments can influence future data being collected. Some people may choose to stop participating in the research. Others may no longer find themselves in the correct demographics for the research. If these factors are not included in the initial design of the research, then it could invalidate the findings that are produced.

2. It takes time. Researchers involved with longitudinal studies may never see the full outcome of their work. It may take several years before the data begins producing observable patterns or relationships that can be tracked. That means the ability to maintain open lines of communication with all researchers is vitally important to the eventual success of the study.

3. The data gathered by longitudinal studies is not always accurate or reliable. It only takes one piece of unreliable or inaccurate data to possibly invalidate the findings that the longitudinal studies produce. Because humans have their own personal bias toward certain subjects, the researcher processing the data may unconsciously alter the data to produce intended results.

4. It relies on the skills of the researchers to be complete. Because data collection occurs in real-time and relies heavily on the skills of the researchers who are tasked with this job, the quality of the data is heavily reliant on those skills. Two different researchers with varying skill levels can produce very different data points from the same subject material. Personal views of the data being collected can also impact the results on both ends, from the subject or the collector.

5. Large sample sizes are required to make the research meaningful. To develop relationships or patterns, a large amount of data must be collected and then mined to create results. That means a large sample size is required so the amount of data being collected can meet expectations. When the subjects being studied are people, it can be difficult to find enough people who are willing to honestly participate in the longitudinal studies.

6. There is a direct cost that is higher than other forms of research. Longitudinal research requires a larger sample size, which means there is a larger cost involved in contacting subjects to collect data. It is also a long-term form of research, which means the costs of the study will be extended for years, or decades, when other forms of research may be completed in a fraction of that time.

7. One person can change a long-term outcome. Because there is such a reliance on individual interpretations within longitudinal studies, it is possible for one person to inadvertently alter or invalidate the data being collected. It is entirely possible for decades of research to be invalidated because one subject or researcher was misleading.

The advantages and disadvantages of longitudinal studies show us that there is a tremendous value available in the ability to find long-term patterns and relationships. If the unpredictable factors of this research format can be planned for in advance and steps taken to remove bias, the data collected offers the potential to dramatically change the fields of medicine, psychology, or sociology.

1.6: Longitudinal Research

Chapter 1: research methods, chapter 2: the social self, chapter 3: social judgement and decision-making, chapter 4: understanding and influencing others, chapter 5: attitudes and persuasion, chapter 6: close relationships, chapter 7: stereotypes, prejudice, and discrimination, chapter 8: helping and hurting, chapter 9: group dynamics.

The JoVE video player is compatible with HTML5 and Adobe Flash. Older browsers that do not support HTML5 and the H.264 video codec will still use a Flash-based video player. We recommend downloading the newest version of Flash here, but we support all versions 10 and above.

Sometimes the goal of a psychological study may be to understand how people’s attitudes and behaviors change over time, or to determine what factors may predict future abilities.

These objectives can be accomplished using a longitudinal design —a research study where data are repeatedly collected from the same group of individuals for a period of time, whether it’s as short as a few weeks or months or as long as several decades.

For example, if a researcher wants to know whether college students’ exercise routines change over the course of their first semester of college, she can use a longitudinal approach and ask students to repeatedly report their workout regiments. She may find that as students get more caught up in their studies, they go to the gym less often.

In addition, the same researcher may keep track of a group of people for twenty years, because she wants to explore how their exercise routines shift across their 20s, 30s and 40s. This approach allows her to best measure changes, within individuals, over time.

In this case, she may discover that those who enjoyed running outdoors in their 20s maintain blood pressure levels, display low amounts of stress, and are more likely to do yoga in their 40s.

While longitudinal research can provide informative results, the method also has its drawbacks. For instance, long-running studies can be very expensive and require a significant time-commitment from the research team and their participants.

Because of this commitment, attrition rates tend to be higher—meaning, more participants dropout. For this reason, the researcher would have to recruit more individuals at the start of the study, expecting a certain number to dropout. Attrition may also cause the study’s sample to be less representative of the population.

Despite its disadvantages, longitudinal research has the power to help us understand variation across human development and the lifespan.

One of the longest-running studies—following people over 80 years as opposed to comparing different groups at different ages—provides a robust measure of human growth—even revealing factors, like close relationships, that lead to people living healthy and happy lives.

Sometimes we want to see how people change over time, as in studies of human development and lifespan. When we test the same group of individuals repeatedly over an extended period of time, we are conducting longitudinal research. Longitudinal research is a research design in which data-gathering is administered repeatedly over an extended period of time. For example, we may survey a group of individuals about their dietary habits at age 20, retest them a decade later at age 30, and then again at age 40.

Let's consider another example. In recent years there has been significant growth in the popular support of same-sex marriage. Many studies on this topic break down survey participants into different age groups. In general, younger people are more supportive of same-sex marriage than are those who are older (Jones, 2013). Does this mean that as we age we become less open to the idea of same-sex marriage, or does this mean that older individuals have different perspectives because of the social climates in which they grew up? Longitudinal research is a powerful approach because the same individuals are involved in the research project over time, which means that the researchers need to be less concerned with differences among cohorts affecting the results of their study.

Often longitudinal studies are employed when researching various diseases in an effort to understand particular risk factors. Such studies often involve tens of thousands of individuals who are followed for several decades. Given the enormous number of people involved in these studies, researchers can feel confident that their findings can be generalized to the larger population. The Cancer Prevention Study-3 (CPS-3) is one of a series of longitudinal studies sponsored by the American Cancer Society aimed at determining predictive risk factors associated with cancer. When participants enter the study, they complete a survey about their lives and family histories, providing information on factors that might cause or prevent the development of cancer. Then every few years the participants receive additional surveys to complete. In the end, hundreds of thousands of participants will be tracked over 20 years to determine which of them develop cancer and which do not.

Clearly, this type of research is important and potentially very informative. For instance, earlier longitudinal studies sponsored by the American Cancer Society provided some of the first scientific demonstrations of the now well-established links between increased rates of cancer and smoking (American Cancer Society, n.d.).

As with any research strategy, longitudinal research is not without limitations. For one, these studies require an incredible time investment by the researcher and research participants. Given that some longitudinal studies take years, if not decades, to complete, the results will not be known for a considerable period of time. In addition to the time demands, these studies also require a substantial financial investment. Many researchers are unable to commit the resources necessary to see a longitudinal project through to the end.

Research participants must also be willing to continue their participation for an extended period of time, and this can be problematic. People move, get married and take new names, get ill, and eventually die. Even without significant life changes, some people may simply choose to discontinue their participation in the project. As a result, the attrition rates , or reduction in the number of research participants due to dropouts, in longitudinal studies are quite high and increases over the course of a project. For this reason, researchers using this approach typically recruit many participants fully expecting that a substantial number will drop out before the end. As the study progresses, they continually check whether the sample still represents the larger population, and make adjustments as necessary.

This text is adapted from OpenStax, Psychology. OpenStax CNX.

Get cutting-edge science videos from J o VE sent straight to your inbox every month.

mktb-description

We use cookies to enhance your experience on our website.

By continuing to use our website or clicking “Continue”, you are agreeing to accept our cookies.

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

What is a Longitudinal Study?: Definition and Explanation

In this article, we’ll cover all you need to know about longitudinal research. 

Let’s take a closer look at the defining characteristics of longitudinal studies, review the pros and cons of this type of research, and share some useful longitudinal study examples. 

Content Index

What is a longitudinal study?

Types of longitudinal studies, advantages and disadvantages of conducting longitudinal surveys.

• Longitudinal studies vs. cross-sectional studies

Types of surveys that use a longitudinal study

Longitudinal study examples.

A longitudinal study is a research conducted over an extended period of time. It is mostly used in medical research and other areas like psychology or sociology. 

When using this method, a longitudinal survey can pay off with actionable insights when you have the time to engage in a long-term research project.

Longitudinal studies often use surveys to collect data that is either qualitative or quantitative. Additionally, in a longitudinal study, a survey creator does not interfere with survey participants. Instead, the survey creator distributes questionnaires over time to observe changes in participants, behaviors, or attitudes. 

Many medical studies are longitudinal; researchers note and collect data from the same subjects over what can be many years.

LEARN ABOUT:   Action Research

Longitudinal studies are versatile, repeatable, and able to account for quantitative and qualitative data . Consider the three major types of longitudinal studies for future research:

Panel study: A panel survey involves a sample of people from a more significant population and is conducted at specified intervals for a more extended period. 

One of the panel study’s essential features is that researchers collect data from the same sample at different points in time. Most panel studies are designed for quantitative analysis , though they may also be used to collect qualitative data and unit of analysis .

LEARN ABOUT: Level of Analysis

Cohort Study: A cohort study samples a cohort (a group of people who typically experience the same event at a given point in time). Medical researchers tend to conduct cohort studies. Some might consider clinical trials similar to cohort studies. 

In cohort studies, researchers merely observe participants without intervention, unlike clinical trials in which participants undergo tests.

Retrospective study: A retrospective study uses already existing data, collected during previously conducted research with similar methodology and variables. 

While doing a retrospective study, the researcher uses an administrative database, pre-existing medical records, or one-to-one interviews.

As we’ve demonstrated, a longitudinal study is useful in science, medicine, and many other fields. There are many reasons why a researcher might want to conduct a longitudinal study. One of the essential reasons is, longitudinal studies give unique insights that many other types of research fail to provide. 

Advantages of longitudinal studies

• Greater validation: For a long-term study to be successful, objectives and rules must be established from the beginning. As it is a long-term study, its authenticity is verified in advance, which makes the results have a high level of validity.
• Unique data: Most research studies collect short-term data to determine the cause and effect of what is being investigated. Longitudinal surveys follow the same principles but the data collection period is different. Long-term relationships cannot be discovered in a short-term investigation, but short-term relationships can be monitored in a long-term investigation.
• Allow identifying trends: Whether in medicine, psychology, or sociology, the long-term design of a longitudinal study enables trends and relationships to be found within the data collected in real time. The previous data can be applied to know future results and have great discoveries.
• Longitudinal surveys are flexible: Although a longitudinal study can be created to study a specific data point, the data collected can show unforeseen patterns or relationships that can be significant. Because this is a long-term study, the researchers have a flexibility that is not possible with other research formats.

Additional data points can be collected to study unexpected findings, allowing changes to be made to the survey based on the approach that is detected.

Disadvantages of longitudinal studies

• Research time The main disadvantage of longitudinal surveys is that long-term research is more likely to give unpredictable results. For example, if the same person is not found to update the study, the research cannot be carried out. It may also take several years before the data begins to produce observable patterns or relationships that can be monitored.
• An unpredictability factor is always present It must be taken into account that the initial sample can be lost over time. Because longitudinal studies involve the same subjects over a long period of time, what happens to them outside of data collection times can influence the data that is collected in the future. Some people may decide to stop participating in the research. Others may not be in the correct demographics for research. If these factors are not included in the initial research design, they could affect the findings that are generated.
• Large samples are needed for the investigation to be meaningful To develop relationships or patterns, a large amount of data must be collected and extracted to generate results.
• Higher costs Without a doubt, the longitudinal survey is more complex and expensive. Being a long-term form of research, the costs of the study will span years or decades, compared to other forms of research that can be completed in a smaller fraction of the time.

Longitudinal studies vs. Cross-sectional studies

Longitudinal studies are often confused with cross-sectional studies. Unlike longitudinal studies, where the research variables can change during a study, a cross-sectional study observes a single instance with all variables remaining the same throughout the study. A longitudinal study may follow up on a cross-sectional study to investigate the relationship between the variables more thoroughly.

The design of the study is highly dependent on the nature of the research questions . Whenever a researcher decides to collect data by surveying their participants, what matters most are the questions that are asked in the survey.

Knowing what information a study should gather is the first step in determining how to conduct the rest of the study. 

With a longitudinal study, you can measure and compare various business and branding aspects by deploying surveys. Some of the classic examples of surveys that researchers can use for longitudinal studies are:

Market trends and brand awareness: Use a market research survey and marketing survey to identify market trends and develop brand awareness. Through these surveys, businesses or organizations can learn what customers want and what they will discard. This study can be carried over time to assess market trends repeatedly, as they are volatile and tend to change constantly.

Product feedback: If a business or brand launches a new product and wants to know how it is faring with consumers, product feedback surveys are a great option. Collect feedback from customers about the product over an extended time. Once you’ve collected the data, it’s time to put that feedback into practice and improve your offerings.

Customer satisfaction: Customer satisfaction surveys help an organization get to know the level of satisfaction or dissatisfaction among its customers. A longitudinal survey can gain feedback from new and regular customers for as long as you’d like to collect it, so it’s useful whether you’re starting a business or hoping to make some improvements to an established brand.

Employee engagement: When you check in regularly over time with a longitudinal survey, you’ll get a big-picture perspective of your company culture. Find out whether employees feel comfortable collaborating with colleagues and gauge their level of motivation at work.

Now that you know the basics of how researchers use longitudinal studies across several disciplines let’s review the following examples:

Example 1: Identical twins

Consider a study conducted to understand the similarities or differences between identical twins who are brought up together versus identical twins who were not. The study observes several variables, but the constant is that all the participants have identical twins.

In this case, researchers would want to observe these participants from childhood to adulthood, to understand how growing up in different environments influences traits, habits, and personality.

LEARN MORE ABOUT: Personality Survey

Over many years, researchers can see both sets of twins as they experience life without intervention. Because the participants share the same genes, it is assumed that any differences are due to environmental analysis , but only an attentive study can conclude those assumptions.

Example 2: Violence and video games

A group of researchers is studying whether there is a link between violence and video game usage. They collect a large sample of participants for the study. To reduce the amount of interference with their natural habits, these individuals come from a population that already plays video games. The age group is focused on teenagers (13-19 years old).

The researchers record how prone to violence participants in the sample are at the onset. It creates a baseline for later comparisons. Now the researchers will give a log to each participant to keep track of how much and how frequently they play and how much time they spend playing video games. This study can go on for months or years. During this time, the researcher can compare video game-playing behaviors with violent tendencies. Thus, investigating whether there is a link between violence and video games.

Conducting a longitudinal study with surveys is straightforward and applicable to almost any discipline. With our survey software you can easily start your own survey today. 

GET STARTED

MORE LIKE THIS

Taking Action in CX – Tuesday CX Thoughts

Apr 30, 2024

QuestionPro CX Product Updates – Quarter 1, 2024

Apr 29, 2024

NPS Survey Platform: Types, Tips, 11 Best Platforms & Tools

Apr 26, 2024

User Journey vs User Flow: Differences and Similarities

Other categories.

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Uncategorized
• Video Learning Series
• What’s Coming Up
• Workforce Intelligence

Longitudinal Study Designs

• Reference work entry
• First Online: 13 January 2019
• Cite this reference work entry

• Stewart J. Anderson 2  

618 Accesses

Longitudinal study designs are implemented when one or more responses are measured repeatedly on the same individual or experimental unit. These designs often seek to characterize time trajectories for cohorts and individuals within cohorts. Three broad categories of longitudinal designs include (1) repeated measures or growth curve designs, where multiple responses for each individual are observed over time or space under the same intervention or other conditions; (2) crossover designs, where individual responses are measured over sequences of interventions so that individuals each “cross over” from one intervention to another; and (3) follow-up studies, where individuals in a cohort are followed until the time that they either have an “event” (e.g., death, depressive episode) or have not had an event but have no further follow-up information. Longitudinal designs may be either randomized where individuals are randomly assigned into different groups or observational where individuals from different well-defined groups are observed over time. In this chapter, I briefly discuss the nature of each of the three designs above and more deeply explore visualization and some analysis techniques for repeated measures design studies via examples of the analyses of two datasets. I conclude with discussion of recent topics of interest in the modeling of longitudinal data including models for intensive longitudinal data, latent class models, and joint modeling of survival and repeated measures data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Anderson SJ. Biostatistics: a computing approach. Boca Raton: Taylor & Francis Group, LLC; 2011.

Google Scholar  

Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: conceptual, strategic and statistical considerations. J Pers Soc Psychol. 1986;51(6):1173–82.

Article   Google Scholar  

Brown H, Prescott R. Applied mixed models in medicine. 2nd ed. West Sussex: Wiley; 2006.

Book   Google Scholar  

Chen K, Lei J. Localized functional principal component analysis. J Am Stat Assoc. 2015;110(511):1266–75.

Choi J-I, Anderson SJ, Richards TJ, Thompson WK. Prediction of transplant-free survival in idiopathic pulmonary fibrosis patients using joint models for event times and mixed multivariate longitudinal data. J Appl Stat. 2014;41(10):2192–205.

Diggle PJ, Heagerty P, Liang K-Y and Zeger SL. Analysis of longitudinal data. 2nd ed. Oxford: Oxford University Press; 2002.

Fitzmaurice G, Davidian M, Verbeke G, Molenberghs G, editors. Longitudinal data analysis. Chapman & Hall/Taylor & Francis Group: Boca Raton; 2009.

Fleiss JL. The design and analysis of clinical experiments. New York: Wiley; 1986.

Grizzle JE. The two–period change–over design and its use in clinical trials. Biometrics. 1965;21:467–80.

Grizzle JE, Allen DM. Analysis of growth and dose response curves. Biometrics. 1969;25(2):357–81.

Guo X, Carlin BP. Separate and joint modeling of longitudinal and event time data using standard computer packages. Am Stat. 2004;58:16–24.

Hardin JW, Hilbe JM. Generalized estimating equations. Boca Raton: Chapman & Hall/CRC; 2003.

Harville D. Maximum likelihood estimation of variance components and related problems. J Am Stat Assoc. 1977;72:320–40.

Hastie T, Tibsharani R, Friedman J. The elements of statistical learning. 2nd ed. New York: Springer; 2009.

Hedeker D, Gibbons RD. Longitudinal data analysis. Hoboken: Wiley; 2006.

Henderson R, Diggle P, Dobson A. Joint modelling of longitudinal measurements and event time data. Biostatistics. 2000;1:465–80.

Jennrich RI, Schlucter MD. Unbalanced repeated-measures models with structured covariance matrices. Biometrics. 1986;42:805–20.

Jones RH, Ackerson LM. Unequally spaced longitudinal data with serial correlation. Biometrika. 1990;77:721–31.

Kalbfleisch JD, Prentice RL. The statistical analysis of failure time data. 2nd ed. New York: Wiley; 2002.

Klein JP, Moeschberger ML. Survival analysis: techniques for censored and truncated data. 2nd ed. New York: Springer; 2003.

Kraemer HC. Discovering, comparing, and combining moderators of treatment on outcome after randomized clinical trials: a parametric approach. Stat Med. 2013;32:19.

Laird NM, Ware JH. Random effects models for longitudinal data. Biometrics. 1982;38:963–74.

Lavori PW, Dawson R. A design for testing clinical strategies: biased adaptive within-subject randomization. J R Stat Soc A. 2000;163:29–38.

Lenze EJ, Mulsant BH, Blumberger DM, Karp JF, Newcomer JW, Anderson SJ, Dew MA, Butters M, Stack JA, Begley AE, Reynolds CF. Efficacy, safety, and tolerability of augmentation pharmacotherapy with aripiprazole for treatment-resistant depression in late life: a randomized placebo-controlled trial. Lancet. 2015;386:2404–12.

Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73:13–22.

Little RJA, Rubin DB. Statistical analysis with missing data. 2nd ed. New York: Wiley; 2002.

McCullagh P, Nelder JA. Generalized linear models. London: Chapman and Hall; 1982.

McCullagh P, Nelder JA. Generalized linear models. 2nd ed. London: Chapman and Hall; 1989.

Molengberghs G, Verbeke G. Models for discrete longitudinal data. New York: Springer; 2005.

Muenz LR, Rubinstein LV. Markov models for covariate dependence of binary sequences. Biometrics. 1985;41:91–101.

Murphy SA. Optimal dynamic treatment regimes. J R Stat Soc B. 2003;65(2):331–66.

Murphy SA. An experimental design for the development of adaptive treatment strategies. Stat Med. 2005;24:1455–81.

Nagin DS. Analyzing developmental trajectories: a semiparametric, group-based approach. Psychol Methods. 1999;4(2):139–57.

Pearl J. Causality: models, reasoning and inference. Cambridge: Cambridge University Press; 2000.

Potthoff R, Roy SN. A generalized multivariate analysis of variance model useful especially for growth curve problems. Biometrika. 1964;51(3):313–26.

Rao CR. Some statistical methods for comparison of growth curves. Biometrics. 1958;14(1):17.

Rao CR. The theory of least squares when parameters are stochastic and its application to the analysis of growth curves. Biometrika. 1965;52(3/4):447–58.

Reynolds CF III, Butters MA, Lopez O, Pollock BG, et al. Maintenance treatment of depression in old age: a randomized, double-blind, placebo-controlled evaluation of the efficacy and safety of donepezil combined with antidepressant pharmacotherapy. Arch Gen Psychiatry. 2011;68(1):51–60.

Rizopoulos D. Dynamic predictions and prospective accuracy in joint models for longitudinal and time-to-event data. Biometrics. 2011;67:819–29.

Rizopoulos D. Joint models for longitudinal and time-to-event data, with applications in R. Boca Raton: Chapman and Hall/CRC; 2012.

Roeder, K, Lynch, KG and Nagin, DS. Modeling uncertainty in latent class membership: a case study in criminology. J Am Stat Assoc. 2011;94:766–776.

Rosner B. Fundamentals of biostatistics. 7th ed. Boston: Brooks/Cole; 2010.

Shiffman S, Dunbar MS, Kirchner TR, Li X, Tindle HA, Anderson SJ, Scholl SM, Ferguson SG. Cue reactivity in converted and native intermittent smokers. Nicotine Tob Res. 2015;17(1):119–23.

Song X, Davidian M, Tsiatis AA. A semiparametric likelihood approach to joint modeling of longitudinal and time-to-event data. Biometrics. 2002;58:742–53.

Stone AA, Shiffman S, Atienza AA, Nebeling L, editors. The science of real-time data capture. New York: Oxford University Press; 2007.

Wallis TA, Schafer J, editors. Models for intensive longitudinal data. New York: Oxford Press; 2006.

Ware JH. Linear models for the analysis of longitudinal studies. Am Stat. 1985;39(2):95–101.

Zeger SL, Liang KY. Longitudinal data analysis for discrete and continuous outcomes. Biometrics. 1986;42:121–30.

Zipunnikov V, Greven S, Shou H, Caffo B, et al. Longitudinal high-dimensional principal components analysis with application to diffusion tensor imaging of multiple sclerosis. Ann Appl Stat. 2014;8(4):2175–202.

Download references

Author information

Authors and affiliations.

Department of Biostatistics, University of Pittsburgh Graduate School of Public Health, Pittsburgh, PA, USA

Stewart J. Anderson

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Stewart J. Anderson .

Editor information

Editors and affiliations.

School of Science and Health, Western Sydney University, Penrith, NSW, Australia

Pranee Liamputtong

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this entry

Cite this entry.

Anderson, S.J. (2019). Longitudinal Study Designs. In: Liamputtong, P. (eds) Handbook of Research Methods in Health Social Sciences. Springer, Singapore. https://doi.org/10.1007/978-981-10-5251-4_70

Download citation

DOI : https://doi.org/10.1007/978-981-10-5251-4_70

Published : 13 January 2019

Publisher Name : Springer, Singapore

Print ISBN : 978-981-10-5250-7

Online ISBN : 978-981-10-5251-4

eBook Packages : Social Sciences Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2023 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

What Is a Longitudinal Study?

Tracking Variables Over Time

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Amanda Tust is a fact-checker, researcher, and writer with a Master of Science in Journalism from Northwestern University's Medill School of Journalism.

Steve McAlister / The Image Bank / Getty Images

The Typical Longitudinal Study

Potential pitfalls, frequently asked questions.

A longitudinal study follows what happens to selected variables over an extended time. Psychologists use the longitudinal study design to explore possible relationships among variables in the same group of individuals over an extended period.

Once researchers have determined the study's scope, participants, and procedures, most longitudinal studies begin with baseline data collection. In the days, months, years, or even decades that follow, they continually gather more information so they can observe how variables change over time relative to the baseline.

For example, imagine that researchers are interested in the mental health benefits of exercise in middle age and how exercise affects cognitive health as people age. The researchers hypothesize that people who are more physically fit in their 40s and 50s will be less likely to experience cognitive declines in their 70s and 80s.

Longitudinal vs. Cross-Sectional Studies

Longitudinal studies, a type of correlational research , are usually observational, in contrast with cross-sectional research . Longitudinal research involves collecting data over an extended time, whereas cross-sectional research involves collecting data at a single point.

To test this hypothesis, the researchers recruit participants who are in their mid-40s to early 50s. They collect data related to current physical fitness, exercise habits, and performance on cognitive function tests. The researchers continue to track activity levels and test results for a certain number of years, look for trends in and relationships among the studied variables, and test the data against their hypothesis to form a conclusion.

Examples of Early Longitudinal Study Design

Examples of longitudinal studies extend back to the 17th century, when King Louis XIV periodically gathered information from his Canadian subjects, including their ages, marital statuses, occupations, and assets such as livestock and land. He used the data to spot trends over the years and understand his colonies' health and economic viability.

In the 18th century, Count Philibert Gueneau de Montbeillard conducted the first recorded longitudinal study when he measured his son every six months and published the information in "Histoire Naturelle."

The Genetic Studies of Genius (also known as the Terman Study of the Gifted), which began in 1921, is one of the first studies to follow participants from childhood into adulthood. Psychologist Lewis Terman's goal was to examine the similarities among gifted children and disprove the common assumption at the time that gifted children were "socially inept."

Types of Longitudinal Studies

Longitudinal studies fall into three main categories.

• Panel study : Sampling of a cross-section of individuals
• Cohort study : Sampling of a group based on a specific event, such as birth, geographic location, or experience
• Retrospective study : Review of historical information such as medical records

Benefits of Longitudinal Research

A longitudinal study can provide valuable insight that other studies can't. They're particularly useful when studying developmental and lifespan issues because they allow glimpses into changes and possible reasons for them.

For example, some longitudinal studies have explored differences and similarities among identical twins, some reared together and some apart. In these types of studies, researchers tracked participants from childhood into adulthood to see how environment influences personality , achievement, and other areas.

Because the participants share the same genetics , researchers chalked up any differences to environmental factors . Researchers can then look at what the participants have in common and where they differ to see which characteristics are more strongly influenced by either genetics or experience. Note that adoption agencies no longer separate twins, so such studies are unlikely today. Longitudinal studies on twins have shifted to those within the same household.

As with other types of psychology research, researchers must take into account some common challenges when considering, designing, and performing a longitudinal study.

Longitudinal studies require time and are often quite expensive. Because of this, these studies often have only a small group of subjects, which makes it difficult to apply the results to a larger population.

Selective Attrition

Participants sometimes drop out of a study for any number of reasons, like moving away from the area, illness, or simply losing motivation . This tendency, known as selective attrition , shrinks the sample size and decreases the amount of data collected.

If the final group no longer reflects the original representative sample , attrition can threaten the validity of the experiment. Validity refers to whether or not a test or experiment accurately measures what it claims to measure. If the final group of participants doesn't represent the larger group accurately, generalizing the study's conclusions is difficult.

The World’s Longest-Running Longitudinal Study

Lewis Terman aimed to investigate how highly intelligent children develop into adulthood with his "Genetic Studies of Genius." Results from this study were still being compiled into the 2000s. However, Terman was a proponent of eugenics and has been accused of letting his own sexism , racism , and economic prejudice influence his study and of drawing major conclusions from weak evidence. However, Terman's study remains influential in longitudinal studies. For example, a recent study found new information on the original Terman sample, which indicated that men who skipped a grade as children went on to have higher incomes than those who didn't.

A Word From Verywell

Longitudinal studies can provide a wealth of valuable information that would be difficult to gather any other way. Despite the typical expense and time involved, longitudinal studies from the past continue to influence and inspire researchers and students today.

A longitudinal study follows up with the same sample (i.e., group of people) over time, whereas a cross-sectional study examines one sample at a single point in time, like a snapshot.

A longitudinal study can occur over any length of time, from a few weeks to a few decades or even longer.

That depends on what researchers are investigating. A researcher can measure data on just one participant or thousands over time. The larger the sample size, of course, the more likely the study is to yield results that can be extrapolated.

Piccinin AM, Knight JE. History of longitudinal studies of psychological aging . Encyclopedia of Geropsychology. 2017:1103-1109. doi:10.1007/978-981-287-082-7_103

Terman L. Study of the gifted . In: The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation. 2018. doi:10.4135/9781506326139.n691

Sahu M, Prasuna JG. Twin studies: A unique epidemiological tool .  Indian J Community Med . 2016;41(3):177-182. doi:10.4103/0970-0218.183593

Almqvist C, Lichtenstein P. Pediatric twin studies . In:  Twin Research for Everyone . Elsevier; 2022:431-438.

Warne RT. An evaluation (and vindication?) of Lewis Terman: What the father of gifted education can teach the 21st century . Gifted Child Q. 2018;63(1):3-21. doi:10.1177/0016986218799433

Warne RT, Liu JK. Income differences among grade skippers and non-grade skippers across genders in the Terman sample, 1936–1976 . Learning and Instruction. 2017;47:1-12. doi:10.1016/j.learninstruc.2016.10.004

Wang X, Cheng Z. Cross-sectional studies: Strengths, weaknesses, and recommendations .  Chest . 2020;158(1S):S65-S71. doi:10.1016/j.chest.2020.03.012

Caruana EJ, Roman M, Hernández-Sánchez J, Solli P. Longitudinal studies .  J Thorac Dis . 2015;7(11):E537-E540. doi:10.3978/j.issn.2072-1439.2015.10.63

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Longitudinal Study Design

Julia Simkus

Editor at Simply Psychology

BA (Hons) Psychology, Princeton University

Julia Simkus is a graduate of Princeton University with a Bachelor of Arts in Psychology. She is currently studying for a Master's Degree in Counseling for Mental Health and Wellness in September 2023. Julia's research has been published in peer reviewed journals.

Learn about our Editorial Process

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

A longitudinal study is a type of observational and correlational study that involves monitoring a population over an extended period of time. It allows researchers to track changes and developments in the subjects over time.

What is a Longitudinal Study?

In longitudinal studies, researchers do not manipulate any variables or interfere with the environment. Instead, they simply conduct observations on the same group of subjects over a period of time.

These research studies can last as short as a week or as long as multiple years or even decades. Unlike cross-sectional studies that measure a moment in time, longitudinal studies last beyond a single moment, enabling researchers to discover cause-and-effect relationships between variables.

They are beneficial for recognizing any changes, developments, or patterns in the characteristics of a target population. Longitudinal studies are often used in clinical and developmental psychology to study shifts in behaviors, thoughts, emotions, and trends throughout a lifetime.

For example, a longitudinal study could be used to examine the progress and well-being of children at critical age periods from birth to adulthood.

The Harvard Study of Adult Development is one of the longest longitudinal studies to date. Researchers in this study have followed the same men group for over 80 years, observing psychosocial variables and biological processes for healthy aging and well-being in late life (see Harvard Second Generation Study).

When designing longitudinal studies, researchers must consider issues like sample selection and generalizability, attrition and selectivity bias, effects of repeated exposure to measures, selection of appropriate statistical models, and coverage of the necessary timespan to capture the phenomena of interest.

Panel Study

• A panel study is a type of longitudinal study design in which the same set of participants are measured repeatedly over time.
• Data is gathered on the same variables of interest at each time point using consistent methods. This allows studying continuity and changes within individuals over time on the key measured constructs.
• Prominent examples include national panel surveys on topics like health, aging, employment, and economics. Panel studies are a type of prospective study .

Cohort Study

• A cohort study is a type of longitudinal study that samples a group of people sharing a common experience or demographic trait within a defined period, such as year of birth.
• Researchers observe a population based on the shared experience of a specific event, such as birth, geographic location, or historical experience. These studies are typically used among medical researchers.
• Cohorts are identified and selected at a starting point (e.g. birth, starting school, entering a job field) and followed forward in time. 
• As they age, data is collected on cohort subgroups to determine their differing trajectories. For example, investigating how health outcomes diverge for groups born in 1950s, 1960s, and 1970s.
• Cohort studies do not require the same individuals to be assessed over time; they just require representation from the cohort.

Retrospective Study

• In a retrospective study , researchers either collect data on events that have already occurred or use existing data that already exists in databases, medical records, or interviews to gain insights about a population.
• Appropriate when prospectively following participants from the past starting point is infeasible or unethical. For example, studying early origins of diseases emerging later in life.
• Retrospective studies efficiently provide a “snapshot summary” of the past in relation to present status. However, quality concerns with retrospective data make careful interpretation necessary when inferring causality. Memory biases and selective retention influence quality of retrospective data.

Allows researchers to look at changes over time

Because longitudinal studies observe variables over extended periods of time, researchers can use their data to study developmental shifts and understand how certain things change as we age.

High validation

Since objectives and rules for long-term studies are established before data collection, these studies are authentic and have high levels of validity.

Eliminates recall bias

Recall bias occurs when participants do not remember past events accurately or omit details from previous experiences.

Flexibility

The variables in longitudinal studies can change throughout the study. Even if the study was created to study a specific pattern or characteristic, the data collection could show new data points or relationships that are unique and worth investigating further.

Limitations

Costly and time-consuming.

Longitudinal studies can take months or years to complete, rendering them expensive and time-consuming. Because of this, researchers tend to have difficulty recruiting participants, leading to smaller sample sizes.

Large sample size needed

Longitudinal studies tend to be challenging to conduct because large samples are needed for any relationships or patterns to be meaningful. Researchers are unable to generate results if there is not enough data.

Participants tend to drop out

Not only is it a struggle to recruit participants, but subjects also tend to leave or drop out of the study due to various reasons such as illness, relocation, or a lack of motivation to complete the full study.

This tendency is known as selective attrition and can threaten the validity of an experiment. For this reason, researchers using this approach typically recruit many participants, expecting a substantial number to drop out before the end.

Report bias is possible

Longitudinal studies will sometimes rely on surveys and questionnaires, which could result in inaccurate reporting as there is no way to verify the information presented.

• Data were collected for each child at three-time points: at 11 months after adoption, at 4.5 years of age and at 10.5 years of age. The first two sets of results showed that the adoptees were behind the non-institutionalised group however by 10.5 years old there was no difference between the two groups. The Romanian orphans had caught up with the children raised in normal Canadian families.
• The role of positive psychology constructs in predicting mental health and academic achievement in children and adolescents (Marques Pais-Ribeiro, & Lopez, 2011)
• The correlation between dieting behavior and the development of bulimia nervosa (Stice et al., 1998)
• The stress of educational bottlenecks negatively impacting students’ wellbeing (Cruwys, Greenaway, & Haslam, 2015)
• The effects of job insecurity on psychological health and withdrawal (Sidney & Schaufeli, 1995)
• The relationship between loneliness, health, and mortality in adults aged 50 years and over (Luo et al., 2012)
• The influence of parental attachment and parental control on early onset of alcohol consumption in adolescence (Van der Vorst et al., 2006)
• The relationship between religion and health outcomes in medical rehabilitation patients (Fitchett et al., 1999)

Goals of Longitudinal Data and Longitudinal Research

The objectives of longitudinal data collection and research as outlined by Baltes and Nesselroade (1979):

• Identify intraindividual change : Examine changes at the individual level over time, including long-term trends or short-term fluctuations. Requires multiple measurements and individual-level analysis.
• Identify interindividual differences in intraindividual change : Evaluate whether changes vary across individuals and relate that to other variables. Requires repeated measures for multiple individuals plus relevant covariates.
• Analyze interrelationships in change : Study how two or more processes unfold and influence each other over time. Requires longitudinal data on multiple variables and appropriate statistical models.
• Analyze causes of intraindividual change: This objective refers to identifying factors or mechanisms that explain changes within individuals over time. For example, a researcher might want to understand what drives a person’s mood fluctuations over days or weeks. Or what leads to systematic gains or losses in one’s cognitive abilities across the lifespan.
• Analyze causes of interindividual differences in intraindividual change : Identify mechanisms that explain within-person changes and differences in changes across people. Requires repeated data on outcomes and covariates for multiple individuals plus dynamic statistical models.

How to Perform a Longitudinal Study

When beginning to develop your longitudinal study, you must first decide if you want to collect your own data or use data that has already been gathered.

Using already collected data will save you time, but it will be more restricted and limited than collecting it yourself. When collecting your own data, you can choose to conduct either a retrospective or prospective study .

In a retrospective study, you are collecting data on events that have already occurred. You can examine historical information, such as medical records, in order to understand the past. In a prospective study, on the other hand, you are collecting data in real-time. Prospective studies are more common for psychology research.

Once you determine the type of longitudinal study you will conduct, you then must determine how, when, where, and on whom the data will be collected.

A standardized study design is vital for efficiently measuring a population. Once a study design is created, researchers must maintain the same study procedures over time to uphold the validity of the observation.

A schedule should be maintained, complete results should be recorded with each observation, and observer variability should be minimized.

Researchers must observe each subject under the same conditions to compare them. In this type of study design, each subject is the control.

Methodological Considerations

Important methodological considerations include testing measurement invariance of constructs across time, appropriately handling missing data, and using accelerated longitudinal designs that sample different age cohorts over overlapping time periods.

Testing measurement invariance

Testing measurement invariance involves evaluating whether the same construct is being measured in a consistent, comparable way across multiple time points in longitudinal research.

This includes assessing configural, metric, and scalar invariance through confirmatory factor analytic approaches. Ensuring invariance gives more confidence when drawing inferences about change over time.

Missing data

Missing data can occur during initial sampling if certain groups are underrepresented or fail to respond.

Attrition over time is the main source – participants dropping out for various reasons. The consequences of missing data are reduced statistical power and potential bias if dropout is nonrandom.

Handling missing data appropriately in longitudinal studies is critical to reducing bias and maintaining power.

It is important to minimize attrition by tracking participants, keeping contact info up to date, engaging them, and providing incentives over time.

Techniques like maximum likelihood estimation and multiple imputation are better alternatives to older methods like listwise deletion. Assumptions about missing data mechanisms (e.g., missing at random) shape the analytic approaches taken.

Accelerated longitudinal designs

Accelerated longitudinal designs purposefully create missing data across age groups.

Accelerated longitudinal designs strategically sample different age cohorts at overlapping periods. For example, assessing 6th, 7th, and 8th graders at yearly intervals would cover 6-8th grade development over a 3-year study rather than following a single cohort over that timespan.

This increases the speed and cost-efficiency of longitudinal data collection and enables the examination of age/cohort effects. Appropriate multilevel statistical models are required to analyze the resulting complex data structure.

In addition to those considerations, optimizing the time lags between measurements, maximizing participant retention, and thoughtfully selecting analysis models that align with the research questions and hypotheses are also vital in ensuring robust longitudinal research.

So, careful methodology is key throughout the design and analysis process when working with repeated-measures data.

Cohort effects

A cohort refers to a group born in the same year or time period. Cohort effects occur when different cohorts show differing trajectories over time.

Cohort effects can bias results if not accounted for, especially in accelerated longitudinal designs which assume cohort equivalence.

Detecting cohort effects is important but can be challenging as they are confounded with age and time of measurement effects.

Cohort effects can also interfere with estimating other effects like retest effects. This happens because comparing groups to estimate retest effects relies on cohort equivalence.

Overall, researchers need to test for and control cohort effects which could otherwise lead to invalid conclusions. Careful study design and analysis is required.

Retest effects

Retest effects refer to gains in performance that occur when the same or similar test is administered on multiple occasions.

For example, familiarity with test items and procedures may allow participants to improve their scores over repeated testing above and beyond any true change.

Specific examples include:

• Memory tests – Learning which items tend to be tested can artificially boost performance over time
• Cognitive tests – Becoming familiar with the testing format and particular test demands can inflate scores
• Survey measures – Remembering previous responses can bias future responses over multiple administrations
• Interviews – Comfort with the interviewer and process can lead to increased openness or recall

To estimate retest effects, performance of retested groups is compared to groups taking the test for the first time. Any divergence suggests inflated scores due to retesting rather than true change.

If unchecked in analysis, retest gains can be confused with genuine intraindividual change or interindividual differences.

This undermines the validity of longitudinal findings. Thus, testing and controlling for retest effects are important considerations in longitudinal research.

Data Analysis

Longitudinal data involves repeated assessments of variables over time, allowing researchers to study stability and change. A variety of statistical models can be used to analyze longitudinal data, including latent growth curve models, multilevel models, latent state-trait models, and more.

Latent growth curve models allow researchers to model intraindividual change over time. For example, one could estimate parameters related to individuals’ baseline levels on some measure, linear or nonlinear trajectory of change over time, and variability around those growth parameters. These models require multiple waves of longitudinal data to estimate.

Multilevel models are useful for hierarchically structured longitudinal data, with lower-level observations (e.g., repeated measures) nested within higher-level units (e.g., individuals). They can model variability both within and between individuals over time.

Latent state-trait models decompose the covariance between longitudinal measurements into time-invariant trait factors, time-specific state residuals, and error variance. This allows separating stable between-person differences from within-person fluctuations.

There are many other techniques like latent transition analysis, event history analysis, and time series models that have specialized uses for particular research questions with longitudinal data. The choice of model depends on the hypotheses, timescale of measurements, age range covered, and other factors.

In general, these various statistical models allow investigation of important questions about developmental processes, change and stability over time, causal sequencing, and both between- and within-person sources of variability. However, researchers must carefully consider the assumptions behind the models they choose.

Longitudinal vs. Cross-Sectional Studies

Longitudinal studies and cross-sectional studies are two different observational study designs where researchers analyze a target population without manipulating or altering the natural environment in which the participants exist.

Yet, there are apparent differences between these two forms of study. One key difference is that longitudinal studies follow the same sample of people over an extended period of time, while cross-sectional studies look at the characteristics of different populations at a given moment in time.

Longitudinal studies tend to require more time and resources, but they can be used to detect cause-and-effect relationships and establish patterns among subjects.

On the other hand, cross-sectional studies tend to be cheaper and quicker but can only provide a snapshot of a point in time and thus cannot identify cause-and-effect relationships.

Both studies are valuable for psychologists to observe a given group of subjects. Still, cross-sectional studies are more beneficial for establishing associations between variables, while longitudinal studies are necessary for examining a sequence of events.

1. Are longitudinal studies qualitative or quantitative?

Longitudinal studies are typically quantitative. They collect numerical data from the same subjects to track changes and identify trends or patterns.

However, they can also include qualitative elements, such as interviews or observations, to provide a more in-depth understanding of the studied phenomena.

2. What’s the difference between a longitudinal and case-control study?

Case-control studies compare groups retrospectively and cannot be used to calculate relative risk. Longitudinal studies, though, can compare groups either retrospectively or prospectively.

In case-control studies, researchers study one group of people who have developed a particular condition and compare them to a sample without the disease.

Case-control studies look at a single subject or a single case, whereas longitudinal studies are conducted on a large group of subjects.

3. Does a longitudinal study have a control group?

Yes, a longitudinal study can have a control group . In such a design, one group (the experimental group) would receive treatment or intervention, while the other group (the control group) would not.

Both groups would then be observed over time to see if there are differences in outcomes, which could suggest an effect of the treatment or intervention.

However, not all longitudinal studies have a control group, especially observational ones and not testing a specific intervention.

Baltes, P. B., & Nesselroade, J. R. (1979). History and rationale of longitudinal research. In J. R. Nesselroade & P. B. Baltes (Eds.), (pp. 1–39). Academic Press.

Cook, N. R., & Ware, J. H. (1983). Design and analysis methods for longitudinal research. Annual review of public health , 4, 1–23.

Fitchett, G., Rybarczyk, B., Demarco, G., & Nicholas, J.J. (1999). The role of religion in medical rehabilitation outcomes: A longitudinal study. Rehabilitation Psychology, 44, 333-353.

Harvard Second Generation Study. (n.d.). Harvard Second Generation Grant and Glueck Study. Harvard Study of Adult Development. Retrieved from https://www.adultdevelopmentstudy.org.

Le Mare, L., & Audet, K. (2006). A longitudinal study of the physical growth and health of postinstitutionalized Romanian adoptees. Pediatrics & child health, 11 (2), 85-91.

Luo, Y., Hawkley, L. C., Waite, L. J., & Cacioppo, J. T. (2012). Loneliness, health, and mortality in old age: a national longitudinal study. Social science & medicine (1982), 74 (6), 907–914.

Marques, S. C., Pais-Ribeiro, J. L., & Lopez, S. J. (2011). The role of positive psychology constructs in predicting mental health and academic achievement in children and adolescents: A two-year longitudinal study. Journal of Happiness Studies: An Interdisciplinary Forum on Subjective Well-Being, 12( 6), 1049–1062.

Sidney W.A. Dekker & Wilmar B. Schaufeli (1995) The effects of job insecurity on psychological health and withdrawal: A longitudinal study, Australian Psychologist, 30: 1,57-63.

Stice, E., Mazotti, L., Krebs, M., & Martin, S. (1998). Predictors of adolescent dieting behaviors: A longitudinal study. Psychology of Addictive Behaviors, 12 (3), 195–205.

Tegan Cruwys, Katharine H Greenaway & S Alexander Haslam (2015) The Stress of Passing Through an Educational Bottleneck: A Longitudinal Study of Psychology Honours Students, Australian Psychologist, 50:5, 372-381.

Thomas, L. (2020). What is a longitudinal study? Scribbr. Retrieved from https://www.scribbr.com/methodology/longitudinal-study/

Van der Vorst, H., Engels, R. C. M. E., Meeus, W., & Deković, M. (2006). Parental attachment, parental control, and early development of alcohol use: A longitudinal study. Psychology of Addictive Behaviors, 20 (2), 107–116.

Further Information

• Schaie, K. W. (2005). What can we learn from longitudinal studies of adult development?. Research in human development, 2 (3), 133-158.
• Caruana, E. J., Roman, M., Hernández-Sánchez, J., & Solli, P. (2015). Longitudinal studies. Journal of thoracic disease, 7 (11), E537.

11 Advantages and Disadvantages of Longitudinal Studies

Longitudinal studies are a type of research or survey that primarily uses the method of observation, which entails that they do not involve interfering with the subjects in any means. These studies are also unique in a way that they follow a certain timeline that is entirely dependent on the respondents, which means that data collection could take years depending on the exact timetable put in place. Most of the time, they are used by psychologists who are looking to measure or identify the impact therapy can have over time, involving long time frames and vast amounts of data.

Now, like any other type of method in conducting research, longitudinal studies also come with certain disadvantages, while they offer obvious advantages. Here are important things to take note when planning to use this methodology:

List of Advantages of Longitudinal Studies

1. They are effective in determining variable patterns over time. Because these studies involve the use and collection of data in long periods of time, they can determine patterns efficiently. By using them, it would be possible for researchers to learn more about cause and effect relationships and make connections in a clearer manner. Aside from this, remember that more data over longer periods of time will allow for more concise and better results. These studies are considered highly valid for determining long-term changes and are unique in themselves when it comes to being able to provide useful data about these individual changes.

2. They can ensure clear focus and validity. With a clear focus, longitudinal studies would let us observe how a certain set of circumstances or an end state would come to be. And while it is natural for people not to remember past events, this problem can be addressed by means of actual recording that ensures a high level of validity.

3. They are very effective in doing research on developmental trends. As mentioned above, these studies are often used in psychology to conduct research on developmental trends across life spans. They are used in sociology to study life events throughout lifetimes or generations. This is so because, unlike cross-sectional studies where different individuals with similar characteristics are being compared, longitudinal studies would track the same people, which means that the differences observed in a group will be less likely to be the result of a change or difference in culture across generations.

4. They are more powerful than cross-sectional studies. As they utilize the observation method without manipulating the state of the world, longitudinal studies have been argued to having less power in terms of detecting causal relationships compared with experiments. However, they are known to have more power than cross-sectional studies when it comes to excluding time-invariants and unobserved individual differences and when it comes to observing a certain event’s temporal order, as they use repeated observations at individual levels.

5. They are highly flexible. Longitudinal studies are often observed to allow flexibility to occur. This means that the focus they use can be shifted while researchers are collecting data.

6. They can provide high accuracy when observing changes. With their quality of being the perfect method to conduct research on developmental trends, these studies can make observation of changes more accurate, making them as the usual option in various fields. In medicine, for example, longitudinal studies are used to discover predictors or indicators of certain diseases, while in advertising, they are used to determine changes that a campaign has made in the behavior of consumers who belong to its target audience and have seen the advertisement.

List of Disadvantages of Longitudinal Studies

1. They require huge amounts of time. Time is definitely a huge disadvantage to any longitudinal study, as it typically takes a substantial amount of time to collect all the data that is required. Also, it takes equally long periods to gather results before the patterns can even start to be made.

2. They risk gathering data that is not 100% reliable. While data is collected at multiple points in this method of conducting research, you cannot pre-determine and take into account the observation periods regardless of what happens between these points. Aside from this, respondents would unknowingly change their qualitative responses over time to better suit what they see as the objective of the observer. Generally, the process involved in longitudinal studies will change how respondents and subjects the questions that are being used.

3. They would risk experiencing panel attrition. One of the biggest disadvantages of conducting longitudinal studies is panel attrition. This means that, if researchers are only relying upon the same group of subjects for a research that takes place at certain points in time in years, then there is the possibility that some of the subjects would no longer be able to participate because of various reasons, such as changes in contact details, refusal, incapacity and even death, which cuts down the usable data to be drawn to formulate the conclusion.

4. They require a large sample size. Another disadvantage that makes longitudinal studies not the perfect option to conduct research is that they typically require large sample sizes. So, you must have a large number of cooperating subjects for your research or else it will not realize or be valid.

5. They can be more expensive compared with cross-sectional studies. Cross-sectional studies are known to be more affordable compared with longitudinal studies and are much quicker in reaching an observational conclusion as they use fewer touch points. Considering that they utilize a sample size that is carefully chosen, rather than subsets, the former can also be more of a help in representing entire populations. The former is observed to be very beneficial when it comes to considering a change in policy, unlike the latter.

A lot of researchers encourage and welcome the use of longitudinal data sets, where they can apply and access data via relevant pathways that are set out by the groups that hold such information. However, longitudinal studies also have some limitations. Based on the advantages and disadvantages listed on this article, do you think these method are more helpful to society than not?

ReviseSociology

A level sociology revision – education, families, research methods, crime and deviance and more!

Longitudinal Studies

Longitudinal Studies are studies in which data is collected at specific intervals over a long period of time in order to measure changes over time. This post provides one example of a longitudinal study and explores some the strengths and limitations of this research method.

With a longitudinal study you might start with an original sample of respondents in one particular year (say the year 2000) and then go back to them every year, every five years, or every ten years, aiming to collect data from the same people. One of the biggest problems with Longitudinal Studies is the attrition rate, or the subject dropout rate over time.

The Millennium Cohort Study

One recent example of a Longitudinal study is the Millennium Cohort Study, which stretched from 2000 to 2011, with an initial sample of 19 000 children.

The study tracked children until the age of 11 and has provide an insight into how differences in early socialisation affect child development in terms of health and educational outcomes.

The study also allowed researchers to make comparisons in rates of development between children of different sexes and from different economic backgrounds.

Led by the Centre for Longitudinal Studies at the Institute of Education , it was funded by the Economic and Social Research Council and government departments. The results below come from between 2006 and 2007, when the children were aged five.

Selected Findings

• The survey found that children whose parents read to them every day at the age of three were more likely to flourish in their first year in primary school, getting more than two months ahead not just in language and literacy but also in maths
• Children who were read to on a daily basis were 2.4 months ahead of those whose parents never read to them in maths, and 2.8 months ahead in communication, language and literacy.
• Girls were consistently outperforming boys at the age of five, when they were nine months ahead in creative development – activities like drama, singing and dancing, and 4.2 months ahead in literacy.
• Children from lower-income families with parents who were less highly educated were less advanced in their development at age five. Living in social housing put them 3.2 months behind in maths and 3.5 months behind in literacy.

The strengths of longitudinal studies

• They allow researchers to trace developments over time, rather than just taking a one-off ‘snapshot’ of one moment.
• By making comparisons over time, they can identify causes. The Millennium Cohort study, for example suggests a clear correlation between poverty and its early impact on low educational achievement

The limitations of longitudinal studies

• Sample attrition – people dropping out of the study, and the people who remain in the study may not end up being representative of the starting sample.
• People may start to act differently because they know they are part of the study
• Because they take a long time, they are costly and time consuming.
• Continuity over many years may be a problem – if a lead researcher retires, for example, her replacement might not have the same rapport with respondents.

Related Posts

Explaining Social Class Differences in Educational Through Longitudinal Studies

Share this:

• Share on Tumblr

One thought on “Longitudinal Studies”

• Pingback: Depression leads to more social media usage, not the other way around! – ReviseSociology

Leave a Reply Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed .

Discover more from ReviseSociology

Subscribe now to keep reading and get access to the full archive.

Continue reading

Home » Pros and Cons » 23 Advantages and Disadvantages of Longitudinal Studies

23 Advantages and Disadvantages of Longitudinal Studies

Longitudinal studies are a form of observational research that is used to collect data. When this type of study is performed, a set of data is collected from each subject over a defined period. The same subjects are used for the research, which means the study can sometimes last for months, if not years.

It is the type of research that is most commonly performed when seeking out information in medical, sociological, or psychological arenas.

Here are the top advantages and disadvantages of longitudinal studies to consider when designing a research study.

Top Advantages of Longitudinal Studies

1. longitudinal studies make it easier to find long-term patterns..

Many research studies focus on short-term data alone. That means long-term data may offer patterns or information that cannot be collected. A longitudinal study would be able to collect that long-term data and locate patterns within it that can benefit the field being researched. This occurs because changes can be tracked over time as the same subjects are being used, allowing cause-and-effect relationships to be found.

2. They offer high validity levels through the collected data.

Because longitudinal studies are a long-term research project, there must be extensive policies and procedures in place from the very beginning of the project. These policies and procedures then dictate the direction of the study, requiring researchers to follow their outline. Because every data collection effort follows this establish protocol, longitudinal studies have high levels of validity because there is a certainty to the authenticity of the research. People can have confidence in the conclusions which are generated.

3. It uses observational methods for data collection.

Most longitudinal studies utilize observational data collection methods because of the long-term nature and design of the research. That makes the data much easier to collect when compared to other study formats. It also provides consistent data that can be applied across all developed metrics. Researches can then use this information to exclude differences or outliers in the data, increasing the overall accuracy of the research while excluding their own personal basis.

4. The format limits the number of mistakes that can be made.

Researchers may wish to draw their own conclusions from the data being collected, but the format of longitudinal studies prevents this from happening. That means from the data collection process to the conclusions that are published, the information being presented has a limited number of mistakes contained within it. That allows the data to be used to create needed changes within the researched field immediately, so that the rest of society can benefit from the conclusions.

5. It offers unique and authentic data.

Because longitudinal studies go directly to the source of information for data, what is collected is often authentic and unique. The principles of data collection allow for long-term relationships to be discovered within the data while short-term benefits can still be collected. It essentially offers the best of both worlds to researchers. Using the results from short-term data, researchers can pursue long-term data that may not have been considered when the study was first implemented.

6. There is more flexibility to be found within longitudinal studies.

Longitudinal studies can be designed to track a single data point. They can also be designed to track multiple metrics simultaneously. Researchers can pivot within the study if they find interesting or unique patterns in the data being collected. Data relationships can be pursued to determine if there is something meaningful within that information. At any time, a shift in focus is permitted if an interesting data point is discovered, which allows for the results and conclusions to be more complete when compared to other study formats and styles.

7. Data collection occurs in real-time.

Longitudinal studies happen in real-time, which means the data being collected is reflective of “now.” That makes it possible for researchers to expand questions or follow tangents based on the responses they receive from each individual. It creates more engagement, which builds a relationship between the subject and the researcher, and that can eventually lead to new insights that might have been held back otherwise.

8. It provides for multiple forms of data collection.

Human memory is a fickle beast. Two people can witness the same event and remember it in very different ways. This occurs because everyone has a unique perspective and their own experiences that affect their memory. For that reason, longitudinal studies offer multiple methods of data collection to ensure the accuracy of what is being collected. Video or audio recordings, diaries, journals, blogs – they can all contribute to the database of information that researchers collect, and then mine, to find conclusions that can be reached.

9. Longitudinal studies make an effort not to manipulate the environment.

This is the advantage of the observational approach that is used with this type of study. Instead of manipulating the environment to produce data, researchers simply collect the data that they can detect with their 5 senses. Asking questions may seem like world manipulation to some. What interviews provide is a way to access personalized information. As long as the researcher is proceeding based on what they observe, the manipulation effect is not present within the researcher.

10. It can be used to pursue developmental trends.

Longitudinal research is often used to determine if developmental trends have any future benefit. It is especially important when considering practical application fields like family medicine to determine better methodologies that can be applied to the general population. Researchers can also pursue trends that they observe on their own to see if there is any correlation between the subjects that would explain the information that they are seeing.

11. They can be used to establish a specific sequence of events.

One person who lives to the age of 100 can be an outlier in any community. Their combination of genetics, healthy living, and optimism, unique to them, can help them live long and happy lives. In a small community, having several people reach the age of 100 is no longer an outlier. There is the potential that there is something present within their environment that is encouraging longer life. Longitudinal studies make it possible to establish a specific sequence of events that would lead to such an outcome, offering the opportunity for that sequence to be duplicated elsewhere.

12. It corrects for the cohort effect.

Whenever there is a large group of people coming together for research purposes, you’re going to have a wide variety of individualized time components. These range from their date of birth, to their current age at the time of the study, to their overall net worth. The long-term nature of this type of study helps to correct any effects that might develop because of these differences, which reduces the individuality of the data that is collected.

Top Disadvantages of Longitudinal Studies

1. the format allows one person to influence the outcome of the study..

Longitudinal studies rely on the expertise, creativity, and honesty of individual researchers for authentic conclusions. That reliance on the individual makes it possible for the data to be corrupted or conclusions to be inaccurate. It does not need to be a purposeful manipulation of the data for the outcomes to be falsified. Misinterpreting data can be just as damaging as purposely misleading the collection of the data and the results would be difficult, at first, to be proven invalid.

2. It offers direct costs that are much higher than other research styles.

Larger sample sizes are required for longitudinal research than in other styles. This requirement is in place because more data is necessary in long-term studies to determine if there are relationships or patterns within the data. That means the direct costs of performing a longitudinal study are typically higher when compared to other research methods. More people must be contacted. More data must be examined. More time must be dedicated to the project. Other research formats do not face these challenges.

3. Long-term studies often see sample sizes change over time.

Larger sample sizes are required for a longitudinal study because of the nature of life. People eventually grow old and die. There may be accidents or natural disasters that occur during the data collection period for the study as well. That means some data may need to be excluded from the research to avoid providing false results and it may be difficult to locate the data that must be removed.

4. It can be difficult to locate willing participants.

Would you want to be tracked over the next 20 years of your life about something? There are three groups of people that researchers encounter when creating a longitudinal study. One group are enthusiastic and willing to help in whatever way they can. One group despises the idea of being contacted by researchers on a regular basis for a long time. The final group doesn’t really care and may not offer accurate data to researchers because of their apathy. Researchers look for enthusiastic participants for authentic data and those folks aren’t always easy to find.

5. Longitudinal research relies heavily on the expertise of each researcher.

Data collection in longitudinal research occurs in real-time, but that means it relies on the skills of the researcher to collect. The average researcher didn’t go to school for journalism, so their ability to recognize follow-up questions or focus on specific data points can be limited. The quality of the data that is collected relies on the quality of the expertise that the researcher brings to the project. That means the quality of the data can also vary if multiple researchers are collecting data.

6. It can have questions of data accuracy.

With a longitudinal study, it only requires one inaccurate data point to throw the entire validity of the study into question. That inaccurate data point can even invalidate years of research and it doesn’t have to be a purposeful collection to cause problems. Researchers are human, just like the rest of us. They have a personal bias toward certain subjects or certain data points that influence how they work, whether they realize it or now. It is not uncommon for researchers to draw conclusions about the data they are collecting before they’ve even collected the information.

7. Time is always an issue with a longitudinal study.

Researchers that begin a longitudinal study may not see the results that their study is able to generate. Some studies take several years to complete. One study, reported on by the Harvard Gazette, discusses a longitudinal study that will be 80 years old in 2018. Scientists began tracking the health of 268 Harvard sophomores in 1938 and as of 2017, 19 of them were still alive. The information collected from this study is important, but it has taken nearly a century for the results to be conclusive. As more data is collected, more time and resources must be utilized to go through the data and look for relationships as well.

8. Study participants may not offer authentic information.

Longitudinal studies involve the same participants over a long period of time. What happens to them can affect the data that is collected. It can also influence how future data is collected. All of this is based on a working assumption that the individual involved is being honest and forthright with the researcher. Some people may stop participating altogether. Unless there are policies and procedures in place that can help to prevent these outcomes, the validity of the study could be questioned before all the data is even collected.

9. Funding is a major challenge for longitudinal studies.

The costs of a longitudinal study don’t need to be prohibitive, but there are factors and influences that can make it difficult to justify the expense. Outside of the actual research cost, many studies of this type must budget for incentives or rewards to encourage subjects to continue their participation. Then, since participants are essentially compensated for providing data, researchers must ensure the information is valid because there will always be critics that question its validity. Some see the compensation of participants as a form of bribery.

10. Inaccuracies are common when analyzing data that is collected by this type of study.

Longitudinal studies often seen inaccuracies during the analysis of the data that is collected. These inaccuracies often arise when hypothesis testing is applied to the collected data, much like it would be for other types of research, such as a cross-sectional study. When this occurs with a longitudinal study, the available data is often under-utilized, which increases the chances of a statistical error occurring.

11. The variable being studied may disappear over time.

Even with the best planning process enacted, policies and procedures cannot stop the fact that a studied variable may disappear from the population group being studied. Should such an event occur, the time and money invested into the research study would be wasted because no valid conclusions could be drawn from the collected data.

The advantages and disadvantages of longitudinal studies direct us toward the unique patterns and relationships of life. Data that is collected is authentic and predictable, which allows researchers to draw conclusions from their findings. For this type of study to work properly, however, there must be procedures in place from the very beginning to remove bias, inaccurate data, and other negative influences.

In return, the data collected has the potential to alter our perspectives in a number of fields.

Related Posts:

• 21 Best SMART Goals Examples for Teachers and Educators
• 10 Proven Psychological Pricing Strategies with Examples
• 50 Best Side Hustles for Accountants
• 30 Best Student Action Plan Examples

Longitudinal research strategies: advantages, problems, and prospects

Affiliation.

• 1 Institute of Criminology, Cambridge University, England.
• PMID: 2055872
• DOI: 10.1097/00004583-199105000-00003

The single-cohort, long-term longitudinal survey has many advantages in comparison with a cross-sectional survey in advancing knowledge about offending and other types of psychopathology, notably in providing information about onset and desistance, about continuity and prediction, and about within-individual change. However, the longitudinal survey also has significant problems, notably in confounding aging and period effects, delayed results, achieving continuity in funding and research direction, and cumulative attrition. This paper suggests the use of a multiple-cohort sequential strategy (the "accelerated longitudinal design") as a way of achieving the benefits of the longitudinal method while minimizing the problems in advancing knowledge about the natural history, causes, prevention, and treatment of psychopathological disorders.

Publication types

• Research Support, Non-U.S. Gov't
• Research Support, U.S. Gov't, Non-P.H.S.
• Child Development
• Child, Preschool
• Cohort Studies
• Criminal Psychology
• Cross-Sectional Studies
• Follow-Up Studies
• Infant, Newborn
• Juvenile Delinquency
• Longitudinal Studies
• Mental Disorders / epidemiology*
• Mental Disorders / etiology
• Research Design*

• Open access
• Published: 23 April 2024

Longitudinal validation of cognitive reserve proxy measures: a cohort study in a rural Chinese community

• Hao Chen 1 , 3   na1 ,
• Jin Hu 1   na1 ,
• Shiqi Gui 1 ,
• Qiushuo Li 1 ,
• Jing Wang 1 ,
• Xing Yang 2 &
• Jingyuan Yang 1  

Alzheimer's Research & Therapy volume  16 , Article number:  87 ( 2024 ) Cite this article

290 Accesses

Metrics details

While evidence supports cognitive reserve (CR) in preserving cognitive function, longitudinal validation of CR proxies, including later-life factors, remains scarce. This study aims to validate CR’s stability over time and its relation to cognitive function in rural Chinese older adults.

Within the project on the health status of rural older adults (HSRO), the survey included baseline assessment (2019) and follow-up assessment (2022). 792 older adults (mean age: 70.23 years) were followed up. The confirmatory factor analysis (CFA) was constructed using cognitive reserve proxies that included years of formal education, social support, hobbies, and exercise. We examined the longitudinal validity of the CR factor using confirmatory factor analyses and measurement invariance and explored the association of CR with cognition using Spearman’s correlation and Generalized Estimating Equations (GEE).

The results showed that CR’s CFA structure was stable over time (T0, χ 2 / df : 3.21/2; RMSEA: 0.02, and T1, χ 2 / df : 7.47/2; RMSEA: 0.05) and that it accepted both configural and metric invariance (Δχ 2 / df  = 2.28/3, P  = 0.52). In addition, it was found that CR had a stable positive relationship with cognitive function across time (T0, r  = 0.54; T1, r  = 0.49). Furthermore, longitudinal CR were associated with MMSE ( β  = 2.25; 95%CI  = 2.01 ~ 2.49).

Conclusions

This study provided valuable evidence on the stability and validity of cognitive reserve proxy measures in rural Chinese older adults. Our findings suggested that cognitive reserve is associated with cognitive function over time and highlighted the importance of accumulating cognitive reserve in later life.

Introduction

As the world’s aging population continues to grow and dementia prevalence increases, the prevention and treatment of dementia have become a top priority for society worldwide [ 1 , 2 ]. China, with its large aging population and high prevalence of dementia, faces an urgent need to address this issue [ 3 , 4 ]. Although no treatment is available to slow or stop dementia, prevention of cognitive decline is an important strategy. Cognitive reserve (CR), as a fascinating concept, emphasizing the capacity of lifestyle choices and life events throughout one’s life to positively influence and enhance cognitive processes, thereby bolstering efficiency and flexibility in addressing cognitive decline [ 5 ]. Accumulated evidence indicated that CR could enhance cognitive adaptability and reduce sensitivity to brain aging, pathology, or injury, delaying clinical symptoms [ 5 , 6 , 7 ].

Nevertheless, since CR cannot be directly measured, it is generally operationalized using proxies such as education, occupation, physical exercise, and social activities [ 8 , 9 ]. Even as numerous studies have shown an association between CR-related proxies and cognitive function, there is heterogeneity in the specific proxies used for CR assessment across different populations. These proxy factors may reflect the unique characteristics and contexts of the studied populations. For instance, studies have shown that education alone is associated with cognition in some populations, while other proxies such as leisure activities or occupation may not exhibit a significant relationship [ 10 , 11 ]. This highlights the importance of considering population-specific factors when examining the relationship between CR-related proxies and cognitive function. Researchers found that higher childhood school performance and engagement in complex job environments during adulthood were associated with a reduced risk of dementia [ 12 ]. Another longitudinal study found that higher social support and engagement in leisure activities improve cognitive reserve in old age [ 13 ]. This underscores the importance of exploring CR-related proxies at different stages of life to understand their contributions to cognitive reserve [ 14 ]. The above studies imply that although education and occupation in early life are prerequisites for cognitive reserve in older adults, additional proxy indicators of cognitive reserve in later life may contribute to its enhancement, offering a new perspective for older adults facing declining cognition. However, it is crucial to exercise caution when interpreting changes in proxy measures, as cognitive reserve itself is not directly measurable. These proxy measures serve as indicators but may not fully capture the true changes in cognitive reserve. Therefore, establishing longitudinal measures to track the changes in proxy indicators of cognitive reserve over time and assessing their structural validity remain areas requiring further development. However, there has been little progress in establishing longitudinal measures of the change in proxy measures of CR over time and assessing structural validity [ 14 , 15 , 16 ].

Measurement invariance techniques are often used in the field of psychology to check the stability of latent measures across time, groups, and ethnicities [ 17 ]. It also is a way to enhance the fairness and validity of neurocognitive ability tests, and although this method is well established for use, it has not yet fully realized its potential in cognition [ 18 ]. According to the existing literature, these techniques have not yet been applied in studies of the CR model. In addition, recent studies have shown that older adults with dementia have lower levels of education and lower levels of occupational complexity as well [ 19 ], and rural older adults have worse CR and cognitive function compared to their urban counterparts [ 20 ]. In China, many rural older adults have low levels of education and have only worked in agriculture during their early life. Therefore, validating the longitudinal effectiveness of cognitive reserve in later life is crucial to confirm its value in delaying cognitive decline in this population. To address this knowledge gap, this study aims to investigate cognitive reserve proxy measures in older adults within a rural Chinese community, validate the structural stability of these measures over time, and estimate their relationship with cognitive function.

Materials and methods

Study design and participants.

This is a cohort from the Guizhou rural older adults’ health study (HSRO) in China. The HSRO is a population-based prospective study conducted in Guizhou, China. The data were obtained using multistage cluster sampling; 12 villages were selected, and the baseline survey was conducted from July to August 2019. Participants were eligible if they were 60-year-old community volunteers who had lived in the area for at least 6 months. The study employed a two-wave (T0-T1) longitudinal survey design. This study included 1,654 older adults who were assessed for cognitive reserve-related proxy measures at baseline. In 2022 (T1), 792 individuals participated in the follow-up surveys. The study was approved by the Ethics Committee of Guizhou Medical University, and all the participants signed informed consent.

Measurement

• Cognitive reserve

(CR) is a theoretical framework that aims to understand the protective factors contributing to cognitive abilities in individuals. In our study, we collected data on four proxies of cognitive reserve: years of education, social support, hobbies, and exercise. To analyze these variables, we employed confirmatory factor analysis, which allowed us to construct a latent variable model representing cognitive reserve. This approach helps us examine the relationship between these proxies and their collective influence on cognitive abilities. Education was measured by 1 item; subjects reported the total number of years at school. The Social Support Rating Scale(SSRS), developed by Xiao [ 21 ], was used to measure the amount of social support. It had three dimensions (subjective support, objective support, and support utilization). There were 10 items in SSRS. And seven questions were answered on a four-point Likert scale, while the remaining questions were answered differently (calculating the number of support sources). Participants were asked a series of questions regarding their engagement in various hobbies and activities, including housework, outdoor activities (e.g., fishing, hiking), gardening, reading books and newspapers, raising poultry or livestock, playing cards or mahjong, watching TV and listening to the radio, participating in organized social activities (e.g., square dancing), as well as indicating if they had no hobbies or engaged in other hobbies not mentioned [ 22 ]. The questionnaire comprised ten items, including an item for indicating the absence of hobbies. The number of hobbies was calculated by assigning scores to the remaining items. The exercise component of the questionnaire was designed based on the common exercise durations of 30 and 60 min for the elderly population [ 23 ]. Participants were asked to indicate the amount of time they spent exercising each day using the following response options: (1) never, (2) 0–30 min, (3) 30–60 min, and (4) more than 60 min.

The Chinese version of the Mini-Mental State Examination (MMSE) scale was used to evaluate individuals’ cognition [ 24 , 25 ]. The test includes 11 items, and the scores can immediately reflect global cognition in clinical, research, and community settings. The scores range from 0 to 30. The changes in cognitive function observed during the follow-up period were categorized into two groups: one group with no reduction in cognitive function (Maintenance) and another group with a decline in cognitive function (Decline).

The smoking category was divided into 3 categories: current smoking (defined as a total of > 100 cigarettes smoked in the past year), ever smoking (including quitting smoking for > 6 months) and never smoking. The alcohol consumption category was divided into 3 categories: regular drinking (defined as drinking on an average of ≥ 3–5 days per week in the past year) or ever/occasional drinking (defined as drink on an average of ≤ 1–2 days per week in the past year), and never drink. Participants in the study were asked which chronic diseases they were diagnosed with, and the number of chronic conditions was counted. Boxes are provided to ask participants if they have specific chronic diseases. These listed the chronic diseases included in the questionnaire, such as arthritis, hypertension, cardiovascular disease, stomach disease, cataracts, chronic lung disease, diabetes, asthma, reproductive disorders, and cancer. In addition, space was provided for participants to write down any other chronic diseases that were not listed.

Statistical analysis

Frequency and median (Interquartile Range (IQR), or range) were used to describe demographic characteristics. Non-parametric tests were employed to analyze the data. The Wilcoxon’s signed rank test was utilized for within-group comparisons of continuous variables with repeated measures, such as comparing baseline and follow-up data within the same group. On the other hand, the Marginal Homogeneity (MH) test was used for longitudinal comparisons between different groups, examining the differences in data distributions across different time points.

To capture proxy factor data for cognitive reserve (CR) more accurately, we utilized continuous information as the preferred form. Confirmatory factor analysis (CFA) was conducted separately for the baseline and follow-up assessments to test model fit. CFA of the CR proxy factor structure evaluation produced eight indicators of goodness of fit: Chi square/df, Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), Normed Fit Index (NFI), Incremental Fit Index (IFI), Akaike information criterion (AIC), and Bayes information criterion (BIC). The following cut-off criteria for the fit index were used: (1) NFI > 0.90; (2) IFI > 0.90; (3) TLI > 0.90; (4) CFI > 0.90, and (5) RMSEA < 0.05; (6) Chi square/df < 5 [ 26 ]. For measurement invariance, a longitudinal two-group CFA was performed, testing for four increasingly stringent types of invariances: configuration, metric, scalar, and strict. Configural invariance was satisfied when indicator variables loaded onto the same factors across groups. Metric invariance is satisfied with adequate model fit when factor loadings remain constant across groups. Scalar invariance is satisfied when factor loadings and intercepts are held constant across groups when model fit is adequate. Strict invariance is satisfied when the factor loadings, intercepts, and residuals are constrained to be equal across groups when the model fit is adequate [ 27 ].

The factor scores for CR were obtained using the maximum likelihood method. Longitudinal changes in cognitive function were calculated using the formula ΔMMSE = MMSE T1 - MMSE T0 , where MMSE T1 represents the follow-up Mini-Mental State Examination (MMSE) score and MMSE T0 represents the baseline MMSE score. Similarly, cognitive reserve was calculated as ΔCR = CR T1 - CR T0 , where CR T1 represents the follow-up cognitive reserve score and CR T0 represents the baseline cognitive reserve score. Based on the above results, scores with the ΔMMSE greater than or equal to 0 were divided into the maintenance group, and those less than 0 were divided into the cognitive decline group. Similarly, the ΔCR was categorized into two groups: the group with increased CR (positive ΔCR) and the group with decreased CR (negative ΔCR). The Spearman’s correlation coefficients between CR scores and MMSE scores were calculated, and the Fisher Z method was used to estimate the significance of the difference between the longitudinal correlation coefficients. This test involves comparing the standard error of the difference between the two coefficients to the difference between the coefficients and calculating a Z-score. If the Z-score is larger than a critical value, the difference between the two coefficients is considered statistically significant [ 28 ]. Generalized estimating equations were employed to analyze the longitudinal relationship and interaction effects between cognitive reserve and cognition. The statistical analyses were performed using SPSS (IBM, Armonk, NY, USA, version 22.0) and R software (package: Lavaan, semTools, version 4.2.2).

A total of 792 study subjects entered the follow-up. The mean (SD) age at baseline assessment was 70.23 (5.87) years. There were 318 males and 474 females in the follow-up data; 96.7% of the subjects’ occupations were farmer only, and 81.2% had received less than six years of education (Table S1 ). At baseline and follow-up, there were statistically significant differences in the SSRS scores, the number of hobbies, the time spent being physically active, and the MMSE scores (Table  1 ).

The CR latent variables constituted by the CFA are shown in Figure S1 ; the highest factor loadings were for hobbies (0.69) at baseline and for social support (0.47) at follow-up. There were excellent goodness-of-fit results at T0 (χ 2 / df : 3.21/2; RMSEA: 0.02; CFI: 0.99; TLI: 1.00; NFI: 0.99; IFI: 1.00) and T1 (χ 2 / df : 7.47/2; RMSEA: 0.05; CFI: 0.96; TLI: 0.87; NFI: 0.94; IFI: 0.96) as presented in Table S2 . By adding constraints (Table  2 ; Tables S3 - S6 ), the model passes only configuration invariance and metric invariance (Δ (Metric – Configural model): Δχ 2  = 2.28; Δ df  = 3; ΔRMSEA= -0.012; ΔCFI = 0.003; ΔTLI = 0.028).

CR model factor scores were significantly positively correlated with cognitive function, either at baseline or follow-up (Fig.  1 ). The cognitive maintenance group exhibited a higher positive ΔCR compared to the cognitive decline group (Figure S2 ). When the longitudinal changes in the correlation coefficient between MMSE and cognitive reserve (CR) were examined, no significant difference in the correlation coefficient was seen for either the increased or decreased CR groups (Figure S3 ). Similarly, in Fig.  2 (A, B), no significant longitudinal changes in correlation coefficients were identified in the cognitive decline group. However, in the cognitive maintenance group, a statistically significant difference in the longitudinal correlation coefficient between MMSE and CR was detected ( P  < 0.05). Further age stratification (referenced to baseline age) showed that the correlations between CR and MMSE scores over time were statistically different for subjects ages 60–69 ( N  = 156; T0: r  = 0.51; T1: r  = 0.35) and 70–79 ( N  = 157; T0: r  = 0.63; T1: r  = 0.48) in the cognitive maintenance group (Fig.  2 ). Generalized estimating equations revealed longitudinal associations between CR and cognitive functioning. Further analyses indicated that the relationships between CR and MMSE scores differed significantly across cognitive subgroups. Interactions were also observed with both sex and age (Table  3 ).

Correlation of CR with MMSE scores in two waves of study. Note : T0 for baseline, T1 for follow-up

Comparison of longitudinal correlation coefficients between CR and MMSE in different cognitive groups. Note : ( A ) and ( B ) show the longitudinal comparison of the Z-transformed correlation coefficients of the cognitive maintenance group and the decline group for all subjects; ( C ), ( D ) are longitudinal comparisons for the 60–69 age group; ( E ), ( F ) are longitudinal comparisons for the 70–79 age group; ( G ), ( H ) are longitudinal comparisons for the ≥80 age group. T0 for baseline, T1 for follow-up

This study assessed the longitudinal stability and validity of proxy indicators of cognitive reserve in a rural Chinese community. Building upon Cognitive Reserve (CR) theory, our study identified a set of CR proxies. The Confirmatory Factor Analysis (CFA) model demonstrated a high fit at two separate time points, and the longitudinal structure confirmed the configuration and metric invariance of measurement. Subsequent analysis revealed robust positive correlations between the CR model’s factor scores and cognitive function. Further analysis showed that the factor scores of the CR model were robustly positively correlated with cognitive function. As far as we know, this is the first study to focus on measurement invariance for the longitudinal validation of the assessment CR model.

In recent years, Kartschmit et al. [ 14 ] summarized the shortcomings of currently available CR assessment tools and concluded that it was necessary to extend the investigation to different populations due to their different experiences in terms of CR proxy parameters. People from various cultural and lifestyle backgrounds may have a diverse variety of proxy parameters to improve the CR. This study conducted on older adults in rural communities in China, most of whom have low education and have been farmers all their lives. They may also have the assumption that certain exposures relatively late in life can also contribute to CR. Similarly, studies have shown that occupation is not associated with cognition in subjects with low levels of education in developing countries [ 11 ]. The present investigation found significant high levels of CFA fit indices at baseline and at follow-up. A Healthy Brain Project cohort found consistent results, demonstrating the stability of the longitudinal structure of the CFA in CR [ 16 ]. Moreover, this study applied measurement invariance, aiming to ensure reliable conclusions about real CR changes across time. According to measurement invariance conventions and reporting, this study accepted both configural invariance and metric invariance. However, while full invariance is preferred, it may not always be achieved. This partial invariance could be attributed to various factors, such as changes in the sample composition or modifications in the measures employed [ 29 ]. Similarly, some cognitive-related studies have failed to meet the most stringent invariance steps, finding significant changes in intercepts and residuals over time [ 30 ]. These differences may be attributed to sample characteristics, and there may indeed be real differences across time in the CR model.

Consistent with previous longitudinal studies of CR [ 31 , 32 ], this study supported the theory of the CR model that CR-related proxies were positively associated with cognitive function at either baseline or follow-up. Furthermore, our findings indicated that older adults in the cognitive maintenance group demonstrated a higher ΔCR, suggesting that the long-term accumulation of cognitive reserve may contribute to the preservation of cognitive performance at a stable level. This aligns with previous research suggesting that the maintenance of cognitive function is associated with cognitive reserve [ 33 ]. In the cognitive maintenance group, our findings revealed a notable decrease in the correlation coefficients between CR and MMSE scores over time, including different age groups. This intriguing observation may be aligned with the notion put forth by Montine et al. [ 34 ], suggesting that cognitive reserve “consumption” is manifested in cognitive performance prior to the onset of cognitive decline. Thus, the observed decline in correlation coefficients may indicate the utilization or “consumption” of cognitive reserve resources in maintaining cognitive performance at a stable level. Conversely, in the cognitive decline group, we observed no significant changes in the correlation coefficients between CR and MMSE scores over time. This finding suggests that, in the context of age-related cognitive decline, cognitive networks may undergo complex and dynamic processes involving the recruitment of additional neural resources for compensation. This observation aligns with the hypothesis proposed by Cabeza et al. [ 35 ] and implies that maintenance and compensation mechanisms could potentially occur simultaneously. Notably, the GEE model results similarly demonstrated statistically significant differences in the associations between CR and MMSE across cognitive subgroups, with interactions observed for both age groups and sexes. However, further research is needed to fully elucidate the intricate dynamics of these processes and their impact on the association between cognitive reserve and cognitive function. It is important to acknowledge that the concepts surrounding cognitive reserve remain subjects of ongoing debate and warrant further investigation through longitudinal studies. Furthermore, our study uncovered stable correlation coefficients between CR and MMSE scores in both the groups with increased and decreased CR over time. These intriguing findings suggest that changes in CR accumulation over time may not significantly impact the association with cognitive function. In other words, our results do not support the assumption that a greater accumulation of cognitive reserve necessarily translates to a stronger correlation with cognitive abilities. However, in the CR increased group, the intercept difference in cognitive level at different times was large, compared to the CR decreased group. Whether this is consistent with the existing evidence finding a more rapid rate of exacerbated cognitive decline in subjects with higher reserves requires further follow-up [ 36 ].

In terms of factor loading in longitudinal CFA, social support and hobbies have better factor loading than physical activity across time. Consistent with a cross-sectional study by the China Health and Retirement Project, older adults participating in hobby groups have better cognitive performance [ 37 ]. The low factor loading of physical activity in the reserve model may be related to the fact that older people in rural China spend more than half of each day with sedentary behavior [ 38 ]. While longer daily physical exercise would be expected to have positive effects on cognitive resilience, the influence of sedentary behavior over a significant portion of the day could potentially attenuate these effects. The majority of our participants, although having a background in farming, are not currently engaged in active agricultural work. This demographic shift from active farming to less physically demanding daily activities may contribute to the observed sedentary lifestyle, which is consistent with the lower factor loading of physical activity in our cognitive reserve model. Social support’s higher factor loading compared to physical activity likely results from rural communities’ strong social bonds, providing a steadier and more significant boost to cognitive reserve than inconsistent physical activity in individuals moving away from labor-intensive work. However, these findings should be interpreted with caution, acknowledging the need for further research to unravel the complex interplay between sedentary behavior, cognitive reserve, and the context of rural Chinese older adults. Additionally, the possibility of reverse causation, where cognitive decline might lead to reduced physical and social activities, calls for more in-depth investigation in future studies to clarify these intricate relationships.

A noteworthy strength of our study utilization of latent variables allows capturing more current CR-related factors each time to reduce recall bias and to ensure that CR measurements are valid across life stages. Nevertheless, some limitations of our study should be noted. Firstly, while the cognitive reserve (CR) proxies used in our study are commonly utilized indicators, they may not fully capture the CR in our specific population of rural older adults. This could potentially limit the generalizability of our findings to other populations or settings. Secondly, the small sample size of very old older adults and the limited number of follow-up waves may have affected the statistical power of our analysis and hindered our ability to capture the dynamic changes in CR over time. It is important to acknowledge that a larger sample size and a longer follow-up period would provide a more robust assessment of the relationship between CR and cognitive outcomes. Thirdly, the measurement of cognitive function using tools like the Mini-Mental State Examination (MMSE) is subject to measurement errors and may have ceiling effects, particularly in populations with high baseline cognitive performance. This could limit our ability to detect subtle changes in cognitive performance and affect the accuracy of the observed correlations. Lastly, the absence of cognitive-related biological indicators, such as neuroimaging or biomarkers, and the limited scope of cognitive status measures used in our study may have restricted our comprehensive assessment of cognitive reserve and its association with cognitive function.

In conclusion, this study provided confirmatory evidence of the longitudinal stability and validity of proxy indicators of cognitive reserve in low-educated rural older adults and indicated that cognitive reserve factors correlate with cognitive performance. Our results highlight the importance of proxy variables for late-life CR throughout the lifespan in preserving cognitive function. They play a crucial role in promoting healthy aging among rural Chinese older adults.

Data availability

The datasets that support the findings of this study are available on request from the corresponding author (Jingyuan Yang, e-mail: [email protected]). The data is not publicly available due to privacy or ethical restrictions.

Beard JR, Officer A, de Carvalho IA, Sadana R, Pot AM, Michel JP, Lloyd-Sherlock P, Epping-Jordan JE, Peeters G, Mahanani WR, Thiyagarajan JA, Chatterji S. The world report on ageing and health: a policy framework for healthy ageing. Lancet. 2016;387(10033):2145–54. https://doi.org/10.1016/S0140-6736(15)00516-4 .

Article   PubMed   Google Scholar  

Jia J, Zuo X, Jia XF, Chu C, Wu L, Zhou A, Wei C, Tang Y, Li D, Qin W, Song H, Ma Q, Li J, Sun Y, Min B, Xue S, Xu E, Yuan Q, Wang M, Huang X, Fan C, Liu J, Ren Y, Jia Q, Wang Q, Jiao L, Xing Y, Wu X, China C, Aging Study G. (2016) Diagnosis and treatment of dementia in neurology outpatient departments of general hospitals in China. Alzheimers Dement 12 (4):446–53. https://doi.org/10.1016/j.jalz.2015.06.1892 .

Nichols E, Szoeke CEI, Vollset SE, Abbasi N, Abd-Allah F, Abdela J, Aichour MTE, Akinyemi RO, Alahdab F, Asgedom SW, Awasthi A, Barker-Collo SL, Baune BT, Béjot Y, Belachew AB, Bennett DA, Biadgo B, Bijani A, Bin Sayeed MS, Brayne C, Carpenter DO, Carvalho F, Catalá-López F, Cerin E, Choi J-YJ, Dang AK, Degefa MG, Djalalinia S, Dubey M, Duken EE, Edvardsson D, Endres M, Eskandarieh S, Faro A, Farzadfar F, Fereshtehnejad S-M, Fernandes E, Filip I, Fischer F, Gebre AK, Geremew D, Ghasemi-Kasman M, Gnedovskaya EV, Gupta R, Hachinski V, Hagos TB, Hamidi S, Hankey GJ, Haro JM, Hay SI, Irvani SSN, Jha RP, Jonas JB, Kalani R, Karch A, Kasaeian A, Khader YS, Khalil IA, Khan EA, Khanna T, Khoja TAM, Khubchandani J, Kisa A, Kissimova-Skarbek K, Kivimäki M, Koyanagi A, Krohn KJ, Logroscino G, Lorkowski S, Majdan M, Malekzadeh R, März W, Massano J, Mengistu G, Meretoja A, Mohammadi M, Mohammadi-Khanaposhtani M, Mokdad AH, Mondello S, Moradi G, Nagel G, Naghavi M, Naik G, Nguyen LH, Nguyen TH, Nirayo YL, Nixon MR, Ofori-Asenso R, Ogbo FA, Olagunju AT, Owolabi MO, Panda-Jonas S, Passos VMA, Pereira DM, Pinilla-Monsalve GD, Piradov MA, Pond CD, Poustchi H, Qorbani M, Radfar A, Reiner RC, Robinson SR, Roshandel G, Rostami A, Russ TC, Sachdev PS, Safari H, Safiri S, Sahathevan R, Salimi Y, Satpathy M, Sawhney M, Saylan M, Sepanlou SG, Shafieesabet A, Shaikh MA, Sahraian MA, Shigematsu M, Shiri R, Shiue I, Silva JP, Smith M, Sobhani S, Stein DJ, Tabarés-Seisdedos R, Tovani-Palone MR, Tran BX, Tran TT, Tsegay AT, Ullah I, Venketasubramanian N, Vlassov V, Wang Y-P, Weiss J, Westerman R, Wijeratne T, Wyper GMA, Yano Y, Yimer EM, Yonemoto N, Yousefifard M, Zaidi Z, Zare Z, Vos T, Feigin VL, Murray CJL. Global, regional, and national burden of alzheimer’s disease and other dementias, 1990–2016. Lancet Neurol. 2019;18(1):88–106. https://doi.org/10.1016/S1474-4422(18)30403-4 . A systematic analysis for the global burden of disease study 2016.

Article   Google Scholar  

Chen X, Giles J, Yao Y, Yip W, Meng Q, Berkman L, Chen H, Chen X, Feng J, Feng Z, Glinskaya E, Gong J, Hu P, Kan H, Lei X, Liu X, Steptoe A, Wang G, Wang H, Wang H, Wang X, Wang Y, Yang L, Zhang L, Zhang Q, Wu J, Wu Z, Strauss J, Smith J, Zhao Y. The path to healthy ageing in China: a peking university-lancet commission. Lancet. 2022;400(10367):1967–2006. https://doi.org/10.1016/S0140-6736(22)01546-X .

Article   PubMed   PubMed Central   Google Scholar  

Stern Y. What is cognitive reserve? Theory and research application of the reserve concept. J Int Neuropsychol Soc. 2002;8(3):448–60.

Stern Y. Cognitive reserve in ageing and alzheimer’s disease. Lancet Neurol. 2012;11(11):1006–12. https://doi.org/10.1016/S1474-4422(12)70191-6 .

Mondini S, Madella I, Zangrossi A, Bigolin A, Tomasi C, Michieletto M, Villani D, Di Giovanni G, Mapelli D. Cognitive reserve in dementia: implications for cognitive training. Front Aging Neurosci. 2016;8:84. https://doi.org/10.3389/fnagi.2016.00084 .

Jones RN, Manly J, Glymour MM, Rentz DM, Jefferson AL, Stern Y. Conceptual and measurement challenges in research on cognitive reserve. J Int Neuropsychol Soc. 2011;17(4):593–601. https://doi.org/10.1017/S1355617710001748 .

Xu H, Yang R, Qi X, Dintica C, Song R, Bennett DA, Xu W. Association of lifespan cognitive reserve indicator with dementia risk in the presence of brain pathologies. JAMA Neurol. 2019;76(10):1184–91. https://doi.org/10.1001/jamaneurol.2019.2455 .

Ye Q, Zhu H, Chen H, Liu R, Huang L, Chen H, Cheng Y, Qin R, Shao P, Xu H, Ma J, Xu Y. Effects of cognitive reserve proxies on cognitive function and frontoparietal control network in subjects with white matter hyperintensities: a cross-sectional functional magnetic resonance imaging study. CNS Neurosci Ther. 2022;28(6):932–41. https://doi.org/10.1111/cns.13824 .

Suemoto CK, Bertola L, Grinberg LT, Leite REP, Rodriguez RD, Santana PH, Pasqualucci CA, Jacob-Filho W, Nitrini R. Education, but not occupation, is associated with cognitive impairment: the role of cognitive reserve in a sample from a low-to-middle-income country. Alzheimers Dement. 2022;18(11):2079–87. https://doi.org/10.1002/alz.12542 .

Dekhtyar S, Wang HX, Scott K, Goodman A, Koupil I, Herlitz A. A life-course study of cognitive reserve in dementia–from childhood to old age. Am J Geriatr Psychiatry. 2015;23(9):885–96. https://doi.org/10.1016/j.jagp.2015.02.002 .

Ihle A, Oris M, Baeriswyl M, Zuber S, Cullati S, Maurer J, Kliegel M. The longitudinal relation between social reserve and smaller subsequent decline in executive functioning in old age is mediated via cognitive reserve. Int Psychogeriatr. 2021;33(5):461–7. https://doi.org/10.1017/S1041610219001789 .

Kartschmit N, Mikolajczyk R, Schubert T, Lacruz ME. Measuring cognitive reserve (cr) - a systematic review of measurement properties of cr questionnaires for the adult population. PLoS ONE. 2019;14(8):e0219851. https://doi.org/10.1371/journal.pone.0219851 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Nogueira J, Gerardo B, Santana I, Simoes MR, Freitas S. The assessment of cognitive reserve: a systematic review of the most used quantitative measurement methods of cognitive reserve for aging. Front Psychol. 2022;13:847186. https://doi.org/10.3389/fpsyg.2022.847186 .

Summers MJ, Thow ME, Ward DD, Saunders NL, Klekociuk SZ, Imlach AR, Summers JJ, Vickers JC. Validation of a dynamic measure of current cognitive reserve in a longitudinally assessed sample of healthy older adults: the tasmanian healthy brain project. Assessment. 2019;26(4):737–42. https://doi.org/10.1177/1073191116685806 .

Millsap RE. Statistical approaches to measurement invariance. Routledge; 2012.

Wicherts JM. The importance of measurement invariance in neurocognitive ability testing. Clin Neuropsychol. 2016;30(7):1006–16. https://doi.org/10.1080/13854046.2016.1205136 .

Darwish H, Farran N, Assaad S, Chaaya M. Cognitive reserve factors in a developing country: education and occupational attainment lower the risk of dementia in a sample of Lebanese older adults. Front Aging Neurosci. 2018;10:277. https://doi.org/10.3389/fnagi.2018.00277 .

Chen X, Xue B, Hu Y. Cognitive reserve over life course and 7-year trajectories of cognitive decline: results from China health and retirement longitudinal study. BMC Public Health. 2022;22(1):231. https://doi.org/10.1186/s12889-022-12671-6 .

Xiao S-Y. The theoretical basis and research application of social support rating scale. J Clin Psychiatry. 1994;4(2):98–100.

Google Scholar  

Zheng Z. Twenty years’ follow-up on elder people’s health and quality of life. China Popul Dev Stud. 2020;3(4):297–309. https://doi.org/10.1007/s42379-020-00045-7 .

Wang L, Li S, Wei L, Ren B, Zhao M. The effects of exercise interventions on mental health in Chinese older adults. J Environ Public Health. 2022;2022(7265718). https://doi.org/10.1155/2022/7265718 .

Katzman R, Zhang MY, Ouang Ya Q, Wang ZY, Liu WT, Yu E, Wong SC, Salmon DP, Grant I. A Chinese version of the mini-mental state examination; impact of illiteracy in a Shanghai dementia survey. J Clin Epidemiol. 1988;41(10):971–8. https://doi.org/10.1016/0895-4356(88)90034-0 .

Article   CAS   PubMed   Google Scholar  

Folstein MF, Folstein SE, McHugh PR. Mini-mental state. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–98. https://doi.org/10.1016/0022-3956(75)90026-6 .

West SG, Taylor AB, Wu W. Model fit and model selection in structural equation modeling. Handb Struct Equation Model. 2012;1:209–31.

Hirschfeld G, Brachel R. (2014) Multiple-group confirmatory factor analysis in r–a tutorial in measurement invariance with continuous and ordinal indicators. Practical Assessment, Research and Evaluation 19:7.

Ramseyer GC. Testing the difference between dependent correlations using the fisher z. J Experimental Educ. 1979;47(4):307–10.

Putnick DL, Bornstein MH. Measurement invariance conventions and reporting: the state of the art and future directions for psychological research. Dev Rev. 2016;41:71–90. https://doi.org/10.1016/j.dr.2016.06.004 .

Bertola L, Bensenor IM, Gross AL, Caramelli P, Barreto SM, Moreno AB, Griep RH, Viana MC, Lotufo PA, Suemoto CK. Longitudinal measurement invariance of neuropsychological tests in a diverse sample from the elsa-brasil study. Braz J Psychiatry. 2021;43(3):254–61. https://doi.org/10.1590/1516-4446-2020-0978 .

Li X, Song R, Qi X, Xu H, Yang W, Kivipelto M, Bennett DA, Xu W. Influence of cognitive reserve on cognitive trajectories: role of brain pathologies. Neurology. 2021;97(17):e1695–706. https://doi.org/10.1212/WNL.0000000000012728 .

Peeters G, Kenny RA, Lawlor B. Late life education and cognitive function in older adults. Int J Geriatr Psychiatry. 2020;35(6):633–9. https://doi.org/10.1002/gps.5281 .

Anaturk M, Kaufmann T, Cole JH, Suri S, Griffanti L, Zsoldos E, Filippini N, Singh-Manoux A, Kivimaki M, Westlye LT, Ebmeier KP, de Lange AG. Prediction of brain age and cognitive age: quantifying brain and cognitive maintenance in aging. Hum Brain Mapp. 2021;42(6):1626–40. https://doi.org/10.1002/hbm.25316 .

Montine TJ, Cholerton BA, Corrada MM, Edland SD, Flanagan ME, Hemmy LS, Kawas CH, White LR. Concepts for brain aging: resistance, resilience, reserve, and compensation. Alzheimers Res Ther. 2019;11(1):22. https://doi.org/10.1186/s13195-019-0479-y .

Cabeza R, Albert M, Belleville S, Craik FIM, Duarte A, Grady CL, Lindenberger U, Nyberg L, Park DC, Reuter-Lorenz PA, Rugg MD, Steffener J, Rajah MN. Maintenance, reserve and compensation: the cognitive neuroscience of healthy ageing. Nat Rev Neurosci. 2018;19(11):701–10. https://doi.org/10.1038/s41583-018-0068-2 .

Lee DH, Seo SW, Roh JH, Oh M, Oh JS, Oh SJ, Kim JS, Jeong Y. Effects of cognitive reserve in alzheimer’s disease and cognitively unimpaired individuals. Front Aging Neurosci. 2021;13:784054. https://doi.org/10.3389/fnagi.2021.784054 .

Fu C, Li Z, Mao Z. Association between social activities and cognitive function among the elderly in China: a cross-sectional study. Int J Environ Res Public Health. 2018;15(2). https://doi.org/10.3390/ijerph15020231 .

Han X, Wang X, Wang C, Wang P, Han X, Zhao M, Han Q, Jiang Z, Mao M, Chen S, Welmer AK, Launer LJ, Wang Y, Du Y, Qiu C. Accelerometer-assessed sedentary behaviour among Chinese rural older adults: patterns and associations with physical function. J Sports Sci. 2022;40(17):1940–9. https://doi.org/10.1080/02640414.2022.2122321 .

Download references

Acknowledgements

The authors would like to acknowledge the efforts of the participants who voluntarily gave their time to participate in the study.

The study was supported by the National Natural Science Foundation of China (Grant No. 81860598).

Author information

Hao Chen and Jin Hu contributed equally to this work.

Authors and Affiliations

Department of Epidemiology and Health Statistics, School of Public Health, The Key Laboratory of Environmental Pollution Monitoring and Disease Control, Guizhou Medical University, Guiyang, China

Hao Chen, Jin Hu, Shiqi Gui, Qiushuo Li, Jing Wang & Jingyuan Yang

School of Medicine and Health Management, Guizhou Medical University, Guiyang, China

The Third People’s Hospital of Guizhou Province, Guiyang, China

You can also search for this author in PubMed   Google Scholar

Contributions

HC conceived and wrote the original draft. JH, SG, QL, XY and JW took responsibility for data collection. HC and JH conducted the statistical analysis. JY revised the paper. All authors contributed to the final version of the paper and have read and approved the final manuscript.

Corresponding author

Correspondence to Jingyuan Yang .

Ethics declarations

Ethics approval and consent to participate.

Written informed consent was obtained from each participant before any study procedure was initiated, and the collection of data on human subjects was approved by the medical ethics committee of Guizhou Medical University (approval No. 2018-092). All methods in this study were performed in accordance with the guidelines of the Declaration of Helsinki.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Chen, H., Hu, J., Gui, S. et al. Longitudinal validation of cognitive reserve proxy measures: a cohort study in a rural Chinese community. Alz Res Therapy 16 , 87 (2024). https://doi.org/10.1186/s13195-024-01451-6

Download citation

Received : 03 January 2024

Accepted : 04 April 2024

Published : 23 April 2024

DOI : https://doi.org/10.1186/s13195-024-01451-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Confirmatory factor analyses
• Measurement invariance

Alzheimer's Research & Therapy

ISSN: 1758-9193

• General enquiries: [email protected]

• Open access
• Published: 19 April 2024

A longitudinal cohort study on the use of health and care services by older adults living at home with/without dementia before and during the COVID-19 pandemic: the HUNT study

• Tanja Louise Ibsen 1 ,
• Bjørn Heine Strand 1 , 2 , 3 ,
• Sverre Bergh 1 , 4 ,
• Gill Livingston 5 , 6 ,
• Hilde Lurås 7 , 8 ,
• Svenn-Erik Mamelund 9 ,
• Richard Oude Voshaar 10 ,
• Anne Marie Mork Rokstad 1 , 11 ,
• Pernille Thingstad 12 , 13 ,
• Debby Gerritsen 14 &
• Geir Selbæk 1 , 15 , 16  

BMC Health Services Research volume  24 , Article number:  485 ( 2024 ) Cite this article

231 Accesses

Metrics details

Older adults and people with dementia were anticipated to be particularly unable to use health and care services during the lockdown period following the COVID-19 pandemic. To better prepare for future pandemics, we aimed to investigate whether the use of health and care services changed during the pandemic and whether those at older ages and/or dementia experienced a higher degree of change than that observed by their counterparts.

Data from the Norwegian Trøndelag Health Study (HUNT4 70 + , 2017–2019) were linked to two national health registries that have individual-level data on the use of primary and specialist health and care services. A multilevel mixed-effects linear regression model was used to calculate changes in the use of services from 18 months before the lockdown, (12 March 2020) to 18 months after the lockdown.

The study sample included 10,607 participants, 54% were women and 11% had dementia. The mean age was 76 years (SD: 5.7, range: 68–102 years). A decrease in primary health and care service use, except for contact with general practitioners (GPs), was observed during the lockdown period for people with dementia ( p  < 0.001) and those aged ≥ 80 years without dementia ( p  = 0.006), compared to the 6-month period before the lockdown. The use of specialist health services decreased during the lockdown period for all groups ( p  ≤ 0.011), except for those aged < 80 years with dementia. Service use reached levels comparable to pre-pandemic data within one year after the lockdown.

Older adults experienced an immediate reduction in the use of health and care services, other than GP contacts, during the first wave of the COVID-19 pandemic. Within primary care services, people with dementia demonstrated a more pronounced reduction than that observed in people without dementia; otherwise, the variations related to age and dementia status were small. Both groups returned to services levels similar to those during the pre-pandemic period within one year after the lockdown. The increase in GP contacts may indicate a need to reallocate resources to primary health services during future pandemics.

Trial registration

The study is registered at ClinicalTrials.gov, with the identification number NCT 04792086.

Peer Review reports

In Norway, similar to most European countries [ 1 , 2 , 3 ], the first wave of the COVID-19 pandemic lasted from 12 March to 15 June 2020 [ 4 ]. During this period, strict infection control measures were introduced to minimise the number of infected people. Health and care services were reduced or locked down, because health professionals were transferred to COVID-19-related services, or hospital wards were reserved for COVID-19 patients. Facilities such as day care services were closed to prevent the spread of infection through social contact, and some services were employed with digital technology. People were urged to stay at home to maintain social distancing and prevent the spread of the virus [ 4 ].

The strict infection control measures aimed mainly to prevent people from hospitalisation and/or death by COVID-19. By 13 November 2022 (last published data), Norway recorded 4,399 cumulative COVID-19-related deaths, of which approximately two-thirds occurred in 2022 (in people of an average age of 85.6 years in 2022) [ 5 ]. From March 2020 to March 2021, compared to the mean all-cause mortality from 2016 to 2019 as a reference, Norway recorded significantly lower all-cause mortality than those recorded by other European Union countries [ 6 ], indicating that Norway had a successful public health strategy. The topic being raised in the present paper, is how infection control measures affected the use of health and care services by the older population, to better prepare ourselves for future health crisis like a pandemic.

Older adults are particularly vulnerable to COVID-19 and at a higher risk of hospitalisation and death [ 7 ]. People with dementia are anticipated to have an even higher risk of mortality than that of people without dementia, because of an impaired immune system [ 8 ]. Fearing the virus, some older adults personally imposed strict infection control measures and cancelled scheduled healthcare appointments. A German study, including participants aged ≥ 73 years, has reported that approximately 30% of the participants reduced or cancelled their medical consultations during the first wave of the pandemic [ 1 ]. A qualitative study including participants aged 65–79 years from Portugal, Brazil, and the United Kingdom has reported that the majority refrained from face-to-face contact with their family doctors in the first wave of the pandemic, as it implied using public transport making social distancing difficult [ 2 ]. Some health and care services have been replaced with online or telephone consultations, which have been beneficial for some parts of the population and challenging for others, especially older adults [ 2 , 3 , 9 ].

People with dementia often need health and care services and practical assistance in their homes to manage their everyday lives [ 10 ]. A Norwegian study including 105 caregivers of people with dementia has reported that 60% experienced a reduction or full cessation of formal care during the first wave of the pandemic as the services were cancelled by the service provider [ 11 ]. This is in line with studies from Sweden and the USA, which reported a significant drop in the use of health and care services during this period [ 12 , 13 ]. However, how the use of primary and specialist healthcare services affected older adults, including people with dementia, as society began a cautious reopening after the first wave of the pandemic remains unclear. A study from the USA conducted a predictive analysis for the post-lockdown period (June 2020–October 2021) on inpatient, outpatient, and emergency services. They found that people with mild cognitive impairment (MCI), Alzheimer’s disease, and related dementia experienced greater and more sustained disruptions in primary and specialist health and care service use than those experienced by people without MCI or dementia [ 13 ].

In the present study, we used a large population-based dataset from the Norwegian Trøndelag Health Study (HUNT) [ 14 ], linked to national registry data on primary and specialist health and care services, to investigate whether the use of health and care services changed during the pandemic, and those with older ages and/or dementia had a higher degree of change than that observed in their counterparts.

Study design and setting

We used a longitudinal cohort design, linking participant data on sex, year of birth, and cognitive status from the HUNT4 70 + survey with later registry data on the use of health and care services from 12 September 2018 to 11 September 2021. This time period equals 18 months before- and 18 months after the Norwegian lockdown on 12 March 2020. This 36-month period was grouped into six periods of six months each, including three pre-lockdown periods (pre1, pre2, and pre3), one lockdown period, and two post-lockdown periods (post1 and post2) (Fig.  1 ). We included a longer lockdown period than the generally denoted period from March to June 2020, as the reopening started slowly, and many older adults imposed strict social distancing on themselves. The next period, 12 September 2020 to 11 March 2021 also included periods with restrictions on social life and activity, such as a maximum of five people gathering and recommendations for wearing a face mask where maintaining distance is difficult. In the last period from 12 March to 11 September 2021, all infection control measures were gradually lifted until Norway was completely reopened on 25 September 2021 [ 4 ]. Trøndelag, the county where the study was conducted, followed national infection control regulations.

Flow-chart of the study periods

Participants

The study included participants aged > 70 years in the fourth wave of the HUNT Study (HUNT4 70 +), which took place between September 2017 and March 2019. The HUNT is a population-based study that has invited the entire adult population from the same geographic area, North-Trøndelag, in four waves, first time in 1984 [ 14 ]. As North-Trøndelag does not comprise any large cities, a random sample of people aged > 70 years from a city in Trondheim (212,000 inhabitants) was also invited. In total, 11,675 participants were included, with 9,930 from North-Trøndelag (response rate 51%) and 1,745 from Trondheim (response rate 34%). We do not judge that there is likely to be any systematic bias introduced by the difference in response rates in different municipalities as the people living at home are similar populations.”. The participants answered a questionnaire that included socio-demographic and clinical data, and they attended a comprehensive clinical evaluation by health professionals [ 15 ]. Participants without sufficient information on their cognitive status ( n  = 202) and nursing home residents ( n  = 866) were excluded (Fig.  2 ). The mean age (76 years, SD 5.7 years) of those included was lower than that of those excluded (82 years, SD 7.9) ( p  < 0.001). The study population included 10,607 participants with complete data on cognitive status. We do not have information on dementia status on the population not included in HUNT4 70 + .

Flow-chart of included participants. HUNT4 70 + : The fourth wave of the Trøndelag health study, 70 year and older cohort

Dementia diagnosis

Two specialists from a diagnostic workgroup of nine medical doctors with comprehensive scientific and clinical expertise (geriatrics, old-age psychiatry, or neurology) independently diagnosed each patient with dementia using the Diagnostic and Statistical Manual of Mental Disorders-5 [ 16 ]. Discrepancies were resolved and consensuses were obtained by the involvement of a third expert. During the diagnostic process, the experts had access to all relevant information from the HUNT4 70 + dataset, such as education, function in activities of daily living, neuropsychiatric symptoms, onset and course of cognitive symptoms, cognitive tests (the Montreal Cognitive Assessment (MoCA) scale [ 17 ] and the Word List Memory Task (WLMT) [ 18 ], and structured interviews with the closest family proxy. A more comprehensive description of the diagnostic process has been published [ 15 ].

Health and care services

Data from two national registries were collected for the entire study period, from September 2018 to September 2021. Health and care services used in primary health care were obtained from The Norwegian Registry of Primary Health Care [ 19 ]. This registry includes individual-level data on municipal health services (contacts with general practitioners (GPs), emergency rooms, and physiotherapists) and care services (care, such as home nurses, and practical assistance in the recipient’s home, day care, respite services and short-term nursing home stays, municipal housing, and nursing home admission) [ 20 ]. Information on the use of specialist health services was based on data from the Norwegian Patient Registry (NPR) [ 21 ]. The NPR holds individual-level data on patients’ use of specialist health services (contacts with somatic hospitals, mental health care, and rehabilitation institutions). The NPR also registers whether the contact was an outpatient consultation, hospitalisation, or day-treatment [ 20 ].

Data were analysed using the STATA 16 software [ 22 ]. Participant characteristics are reported as means with SD, frequencies, or percentages, as appropriate. Those who were admitted to a nursing home ( n  = 364) or died ( n  = 821) during the study period were censored and participated in only half of the person-time during the study period. Duplicates were removed (3,293 observations). The mean number of health and care services per person in each period (with 95% confidence interval [CI]) was predicted from a multilevel mixed-effects linear regression model with random intercept, where random effects varied across the individuals. In the regression model, the number of services per person was the outcome variable and sex, age, cognitive status (no dementia/dementia), and period were covariates.Age and cognitive status are relevant confounders to address the aim of the present study, and sex is included as a key sociodemographic measure in epidemiological research. [ 23 , 24 ]. To allow for different time trends by cognitive status group, the interaction term period by cognitive status was included in the regression model. In the predictions, the adjusted variables were fixed at their mean values. The significance level was set at p  < 0.05. To investigate the use of health and care services before and during the pandemic, the number of care services implemented within each period and the number of contacts within each period for primary and specialist health services were aggregated. Hence, for care services, we used the date on which the service was implemented, for example the date on which practical assistance at home was implemented. For health services, we used the date when the service occurred, for example, the date a person had contact with a GP or the date a person had contact with a hospital, either for outpatient consultation, hospitalisation, or day-treatment.

In the Results section, we report significant differences between the lockdown period and all the pre- and post-lockdown periods, and between pre2 and post2, as these periods comprise the same seasonal months, one year before and one year after the lockdown, respectively.

The study included 10,607 participants, of whom 54% were women, and 11% had dementia (Table  1 ). The mean age of the participants on 1 January 2017 was 76 years (SD 5.7, range: 68–102 years), and 7,769 participants (73%) were < 80 years old. During the 36-month follow-up period, the study sample was reduced by 10% (from 10,607 to 9,568) due to censoring for death and/or nursing home admission (Table  2 ). The dropout rate was higher in those with dementia than in those without dementia (37% vs. 7%, p  < 0.001). During these 36-months, the total number of contacts with primary health services was 554,061, which corresponded to 9.2 contacts per person per 6-month period (Table  3 ). People with dementia had more contact with health services in the municipality than the contact made by those without dementia (11.3 vs. 8.8 contacts per person per 6-month period, p  < 0.001). The total number of care services implemented for our study population was 20,411, which corresponded to 0.3 care services per person per 6-month period. People with dementia received more care services than those received by people without dementia (1.2 vs. 0.2 care services per person per 6-month period, p  < 0.001). The total number of contacts with specialist health services was 141,994, which corresponded to 2.3 contacts per person per 6-month period. People with dementia had less contact with specialist health services than the contact made by those without dementia (2.2 vs. 2.6 contacts per person per 6-month period, p  < 0.001).

Primary health and care services

Health services.

During the 36-month study period, contact with GPs was the most used health service (66%), followed by physiotherapy services (30%), and contact with GPs in emergency rooms (4%).

The following model only presents contact with GPs, including GPs in emergency rooms, as contact with GPs was the most frequently used primary health service.

The age- and sex-adjusted model (Fig.  3 ) shows that for those aged < 80 years with dementia, the mean number of GP contacts during the lockdown period was higher than that during pre1 (1.27, p  < 0.001) and pre3 (0.82, p  = 0.002) and lower than that during post1 (1.67, p  < 0.001) and post2 (0.84, p  < 0.002). The mean number of GP contacts during post2 was higher than that during pre2 (0.32, p  < 0.001).

Mean number of registered contacts with general practitioners (GPs) per period, pre-lockdown, during lockdown and post-lockdown, including GPs at emergency rooms, for participants < 80 versus ≥ 80 years, divided in people with- or without dementia. Mean number of contacts was predicted in a mixed-effects linear regression model adjusted by period, cognitive status, sex, age, and the interaction period*cognitive status. In the predictions, the adjustment variables age and sex were fixed at the mean values

For those without dementia, the mean number of GP contacts during the lockdown was higher than that during pre1 (0.45, p  < 0.001) and pre2 (0.51, p  < 0.001) and lower than that during post1 (1.18, p  < 0.001) and post2 (0.59, p  < 0.001). The mean number of GP contacts during post2 was higher than that during pre2 (1.11, p  < 0.001).

For those aged ≥ 80 years with dementia, the mean number of GP contacts during the lockdown was higher than that during pre1 (1.45, p  < 0.001) and pre2 (0.96, p  = 0.015) and lower than that during post1 (2.31, p  < 0.001). The mean number of GP contacts during post2 was higher than that during pre2 (1.72, p  < 0.001).

For those without dementia, the mean number of GP contacts during the lockdown was higher than that during pre1 (1.15, p  < 0.001) and pre2 (0.91, p  < 0.001) and lower than that during post1 (1.86, p  < 0.001) and post2 (0.60, p  < 0.002). The mean number of GP contacts during post2 was higher than that during pre2 (1.51, p  < 0.001).

Care services

During the 36-month study period, care and practical assistance at home represented the largest service group (69%), followed by short-term nursing home stays and respite services (21%), nursing home admissions (4%), municipal housing (3%), and day care services (4%). The following model presents all combined care services.

The age- and sex-adjusted model (Fig.  4 ) shows that for those aged < 80 years with dementia, the mean number of care services implemented during the lockdown was lower than that during pre3 (0.37, p  < 0.001) and post1 (0.43, p  < 0.001). The mean number of care services implemented in post2 was higher than that during pre2 (0.13, p  = 0.039).

Mean number of care services implemented per period, pre-lockdown, during lockdown and post-lockdown, as health care and practical assistance in the home, day- and respite services, short-term institutional stay, and nursing home admission, for participants < 80 versus ≥ 80 years, divided in people with- and without dementia. Mean number of care services implemented was predicted in a mixed-effects linear regression model adjusted by period, cognitive status, sex, age, and the interaction period*cognitive status. In the predictions, the adjustment variables age and sex were fixed at the mean values

For those without dementia, the mean number of care services implemented during the lockdown was higher than that during pre1 (0.5, p  = 0.001) and pre2 (0.04, p  = 0.005) and lower than that during post1 (0.03, p  = 0.044). The mean number of care services implemented during post2 was higher than that during pre2 (0.07, p  < 0.001).

For those aged ≥ 80 years with dementia, the mean number of care services implemented during the lockdown was lower than that during pre3 (0.76, p  < 0.001).

For those without dementia, the mean number of care services implemented during the lockdown was higher than that during pre1 (0.22, p  = 0.001) and pre2 (0.17, p  = 0.011) and lower than that during pre3 (0.18, p  = 0.006) and post1 (0.18, p  = 0.007). The mean number of care services implemented during post2 was higher than that during pre2 (0.24, p  < 0.001).

Specialist health services

During the study period, service provision from somatic hospitals was the most used service (96%), followed by mental health care (3%), and treatment at a rehabilitation institution (1%). Somatic hospital services included outpatient consultations (88%), hospitalisation (9%), and daily treatment (3%). The following model only presents contacts with somatic hospital services, as this is the most frequently used specialist health service.

The age- and sex-adjusted models (Fig.  5 ) show that for those aged < 80 years with dementia, the mean number of contacts with somatic hospital services during the lockdown was lower than that during post1 (0.67, p  = 0.002) and post2 (0.48, p  = 0.025). The mean number of contacts with somatic hospital services in post2 was higher than that during pre2 (0.61, p  = 0.004).

Mean number of registered contacts with somatic hospital services per period, pre-lockdown, during lockdown and post-lockdown, for participants < 80 versus ≥ 80 years, divided in people with- or without dementia. Mean number of contacts was predicted in a mixed-effects linear regression model adjusted by period, cognitive status, sex, age, and the interaction period*cognitive status. In the predictions, the adjustment variables age and sex were fixed at the mean values

For those without dementia, the mean number of contacts with somatic hospital services during the lockdown was lower than that during pre1 (0.16, p  = 0.002), pre3 (0.40, p  < 0.001), post1 (0.43, p  < 0.001), and post2 (0.34, p  < 0.001). The mean number of contacts with somatic hospital services in post2 was higher than that during pre2 (0.25, p  < 0.001).

For those aged ≥ 80 years with dementia, the mean number of contacts with somatic hospital services during the lockdown was lower than that during pre2 (0.54, p  = 0.003), pre3 (0.46, p  = 0.011), post1 (0.44, p  = 0.022), and post2 (0.42, p  = 0.040).

For those without dementia, the mean number of contacts with somatic hospital services during the lockdown was lower than that during pre3 (0.49, p  < 0.001), post1 (0.41, p  < 0.001), and post2 (0.41, p  < 0.001). The mean number of contacts with somatic hospital services in post2 was higher than that during pre2 (0.29, p  = 0.001).

This population-based study revealed that people with dementia experienced a larger decrease in the use of primary care services implemented during the lockdown than that experienced by people without dementia. Contact with GPs was maintained at a normal level or increased in both groups during the lockdown. The use of specialist health services decreased in both groups during the lockdown period except for those aged < 80 years with dementia. The use of primary health and care services, and specialist health services was at the same or higher-level post-lockdown (post2) as pre-lockdown (pre2). Collectively, these results indicate an increased burden on primary health services during the lockdown.

Both cognitive groups had a similar number of GP contacts during lockdown as pre-lockdown. Those aged < 80 years with dementia experienced an increased number of GP contacts during the lockdown compared to the numbers during the 6-month period before the lockdown (pre3). Furthermore, all the groups had an increased number of GP contacts in the first 6-months period post-lockdown (post1). Unfortunately, we were unable to identify whether the consultations were digital in our material; however, digital consultations may have contributed to maintaining contact with GPs during the pandemic. This corresponds with the results of a previous study which has reported that the Norwegian population experienced an increased use of telephone and video consultations during the pandemic [ 3 ]. However, a survey during the pandemic in the same study population as that of the present study (HUNT4 70 +) revealed that only 8% reported contact with healthcare professionals via screen-based media or telephone at least once a month during the pandemic [ 9 ]. In addition, a survey of video consultations among Norwegian GPs during the pandemic revealed that video consultations were unsuitable for the oldest population [ 25 ].

The results of the present study may indicate that GPs managed to serve older adults in Norway during the pandemic and that the cancellations of medical consultations described among older adults in other countries [ 1 , 2 ] have been less extensive in Norway. Meanwhile, contact with GPs may have shifted towards more severe cases, where patients in need of specialist health services who postponed contact because of COVID-19 used the primary care service. In addition, the increase in GP contact post-lockdown may imply an increased stress level among older adults and an increase in health problems during the lockdown, which will be discussed in more detail in a later section.

Our finding that people with dementia experienced a larger decrease in the number of care services implemented during the lockdown than that experienced by people without dementia is in line with those of earlier studies [ 11 , 13 ]. This is most likely a consequence of the fact that people with dementia use care services more often and thus, are more affected when such services are reduced or locked down. Interestingly, those with dementia in both age groups experienced a significant increase in new services implemented in the 6-month period before the lockdown (pre3). However, the possible cause for the increase in care services implemented, such as a reduction in other services or societal changes during this period, remains unconfirmed. The most likely explanation is an increase in service needs related to dementia progression, although some random fluctuations cannot be ruled out.

Care service providers have reported a deterioration in older adults’ health during the pandemic related to the absence of social support, which, in turn, has led to less support with meals, practical help, and physical activity [ 26 ]. Next of kin reported that people with dementia had a reduction in cognitive- and functional abilities because of the limited possibility of meaningful activities and mental stimulation when they had to stay at home [ 27 , 28 ]. Furthermore, a lack of social connections [ 29 ] and perceived social support [ 30 ] are associated with cognitive decline and depression. Based on these findings, it can be assumed that the need for care services may be the same or higher post-lockdown than that in the 6-month period before the pandemic (pre3). However, the number of care services implemented post-lockdown (post2) was at the same level as that at pre-lockdown (pre2).

This study revealed that somatic hospital services for those aged ≥ 80 years were the only services with a lower level of contact during the lockdown period than during the comparable pre-lockdown period (pre2). Both those with and without dementia had a decrease in somatic hospital services during the lockdown period, compared to the 6-months period before the lockdown. This corresponds with findings from an Italian study conducted in the autumn of 2020, reporting that hospitalisations and outpatient visits among older adults aged ≥ 65 years were reduced by 18.3% during the pandemic [ 31 ].

The decrease in the use of somatic hospital services during the lockdown observed in the present study was most likely related to strict infection control measures that prevented a widespread COVID-19 outbreak. Furthermore, it may be interpreted as a precautionary measure taken to minimize the risk of exposing older adults to hospitals, where a considerable number were affected by COVID-19. Hospital services experienced the greatest decline in activity during the lockdown due to preparedness for COVID-19 patients [ 32 ]. In the present study, all the groups returned to the same or a higher level of contact with somatic hospital services post-lockdown (post2), than they had pre-lockdown (pre2). Conversely, a study from the USA has suggested that people with dementia or MCI would experience more sustained disruption in primary and specialist health services than that experienced by people without such diagnoses [ 13 ]. Another study from the USA has revealed that those with comorbidities, often present among people with dementia, were at a higher risk of delayed or missed care during the pandemic [ 33 ]. The contrast in the findings may be related to differences in the healthcare system. In addition, the World Health Organization has reported disruptions in both primary and specialist health services worldwide two years into the pandemic. High-income countries reported fewer service disruptions than those reported by low-income ones [ 34 ]. The increase in GP contact post-lockdown in the present study may indicate that primary health services have been able to relieve specialist health services in Norway, so that people with dementia and others in need of specialist health services may be prioritised.

The variation in the frequency of contact with both somatic hospital services and GPs may be observed in the context of normal seasonal variations, where contact might be higher in the autumn and winter months (pre1, pre3, and post2) than in the spring and summer months (pre2, lockdown, and post2). However, the Norwegian Institute of Public Health has reported that the seasonal flu outbreak from December 2019 to March 2020, which corresponds with the 6-month period before the lockdown (pre3), was limited compared to those in previous years [ 35 ]. Thus, normal variations due to seasonal flu cannot provide a full explanation for more contact with GPs and somatic hospital services in the 6-month period before lockdown (pre3). The next seasonal flu, expected from December 2020 to March 2021 (post1), did not appear as expected, most likely because of the infection control measures in connection with the COVID-19 outbreak [ 36 , 37 ]. The increase in the frequency of contact with GPs and somatic hospital services detected in the 6-month period after the lockdown (post1) may be explained by the fact that people had less contact with these services for diseases other than COVID-19 during the first wave of the pandemic [ 32 ], and that these consultations accumulated when society started reopening. Furthermore, the increase in contact with GPs and somatic hospital services after the lockdown may be explained by the increased contact between people, which may have caused an increased spread of infections [ 37 ].

Finally, the increase in mental health problems during the pandemic [ 27 , 28 , 30 ], may have required additional medical supervision. Studies have reported an increase in depression among older adults during the pandemic, a related increase in the prescription of antidepressant medication [ 30 , 38 ], and the need for primary health services, such as GPs, and specialist services, such as hospital services [ 38 ].

Strength and limitations

The main strength of the present study is its large population-based survey sample merged with unique national registry data on primary and specialist health care services. This provided objective data regarding the participants’ service use. Despite the large study sample, all the participants were from the middle region of Norway, which may differ from the population in other parts of the country and outside Norway. Furthermore, the study sample was a homogenous group of participants mainly born in Norway, and the results cannot be generalised to other ethnic groups. Although the diagnostic process for dementia was thorough, the diagnosis was based on collected research data without access to imaging or biomarker data which may have caused misclassification. As our goal was to estimate the actual change in service use based on dementia status among younger and older adults, the analysis does not include health-related covariates such as comorbidity and functional level. Finally, the information on dementia status was collected from 2017 to 2019 and may have changed during the study period from September 2018 to September 2021.

The use of primary care and specialist health services was immediately reduced during the COVID-19 lockdown period. Within primary care services, people with dementia experienced a more pronounced reduction than that experienced by people without dementia; however, age and dementia status only demonstrated small variations. One year after the lockdown, service provisions returned to a level similar to or higher than that of one year before the lockdown for all groups. Our findings indicate that infection control and management limited the scope of action within care services and specialist health services during the lockdown, leaving GPs on the front line to manage medical problems and psychological stress in the population. In any future pandemic, the reallocation of resources for primary health services could make us better equipped to meet the needs of the population.

Availability of data and materials

The data that support the findings of this study are available from the HUNT database and the Norwegian registry database, Helsedata, but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of the HUNT database and the Norwegian registry database Helsedata.

Brandl C, Zimmermann ME, Günther F, Dietl A, Küchenhoff H, Loss J, Stark KJ, Heid IM. Changes in healthcare seeking and lifestyle in old aged individuals during COVID-19 lockdown in Germany: the population-based AugUR study. BMC Geriatr. 2022;22(1):34.

Article   CAS   PubMed   PubMed Central   Google Scholar  

von Humboldt S, Low G, Leal I. Health Service Accessibility, Mental Health, and Changes in Behavior during the COVID-19 Pandemic: A Qualitative Study of Older Adults. Int J Environ Res Public Health. 2022;19(7):4277.

Article   Google Scholar  

Tu K, Sarkadi Kristiansson R, Gronsbell J, de Lusignan S, Flottorp S, Goh LH, Hallinan CM, Hoang U, Kang SY, Kim YS, et al. Changes in primary care visits arising from the COVID-19 pandemic: an international comparative study by the International Consortium of Primary Care Big Data Researchers (INTRePID). BMJ Open. 2022;12(5):e059130.

Article   PubMed   Google Scholar  

Governmentof Norway: Timeline: News from Norwegian Ministries about the Coronavirus disease Covid-19 [ https://www.regjeringen.no/no/tema/Koronasituasjonen/tidslinje-koronaviruset/id2692402/ ]. Accessed 12 May 2023.

Norwegian Institute of Public Health: Dødelighet i Norge under koronapandemien 2020 til høsten 2022 [Mortality in Norway during the corona pandemic 2020 to autumn 2022] [ https://www.fhi.no/publ/2022/Dodelighet-under-pandemien/ ]. Accessed 22 June 2023.

Statistics Norway: Hvordan gikk det? Korona i Norge og EU [How did it go? Corona in Norway and EU] [ https://www.ssb.no/helse/faktaside/konsekvenser-av-korona ]. Accessed 22 June 2023.

Ho FK, Petermann-Rocha F, Gray SR, Jani BD, Katikireddi SV, Niedzwiedz CL, Foster H, Hastie CE, Mackay DF, Gill JMR, et al. Is older age associated with COVID-19 mortality in the absence of other risk factors? General population cohort study of 470,034 participants. PLoS ONE. 2020;15(11):e0241824.

Bianchetti A, Rozzini R, Guerini F, Boffelli S, Ranieri P, Minelli G, Bianchetti L, Trabucchi M. Clinical Presentation of COVID19 in Dementia Patients. J Nutr Health Aging. 2020;24(6):560–2.

Eriksen SRA, Selbæk G, Bjørkløf G, Tveito M, Bergh S, Langhammer A, Næss M, Ibsen T. Bruk av skjermbaserte medier blant eldre under covid-19-pandemien En HUNT-studie [Use of screen-based media among older people during the COVID-19 pandemic A HUNT study]. Sykepleien Forskning. 2022;17(88131):e-88131.

Google Scholar  

Bradbury KM, Moody E, Aubrecht K, Sim M, Rothfus M. Equity in Changes to Dementia Care in the Community during the First Wave of the COVID-19 Pandemic in High Income Countries: A Scoping Review. Societies. 2022;12(2):30.

Vislapuu M, Angeles RC, Berge LI, Kjerstad E, Gedde MH, Husebo BS. The consequences of COVID-19 lockdown for formal and informal resource utilization among home-dwelling people with dementia: results from the prospective PAN.DEM study. BMC Health Serv Res. 2021;21(1):1003.

Article   PubMed   PubMed Central   Google Scholar  

Ekman B, Arvidsson E, Thulesius H, Wilkens J, Cronberg O. Impact of the Covid-19 pandemic on primary care utilization: evidence from Sweden using national register data. BMC Res Notes. 2021;14(1):424.

Tannous J, Pan A, Bako A, Potter T, Jones SL, Janjan N, Smith ML, Seshadri S, Ory MG, Vahidy FS. COVID-19 associated disruptions in routine health care of people with mild cognitive impairment or dementia. Alzheimers Dement (Amst). 2022;14(1):e12323.

Åsvold BO, Langhammer A, Rehn TA, Kjelvik G, Grøntvedt TV, Sørgjerd EP, Fenstad JS, Heggland J, Holmen O, Stuifbergen MC, Vikjord SAA, Brumpton BM, Skjellegrind HK, Thingstad P, Sund ER, Selbæk G, Mork PJ, Rangul V, Hveem K, Næss M, Krokstad S. Cohort Profile Update: The HUNT Study. Norway Int J Epidemiol. 2023;52(1):80–91.

Gjøra L, Strand BH, Bergh S, Borza T, Brækhus A, Engedal K, Johannessen A, Kvello-Alme M, Krokstad S, Livingston G, et al. Current and Future Prevalence Estimates of Mild Cognitive Impairment, Dementia, and Its Subtypes in a Population-Based Sample of People 70 Years and Older in Norway: The HUNT Study. J Alzheimers Dis. 2021;79(3):1213–26.

American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-5. Warshington, DC: American Psychiatric Association; 2013.

Book   Google Scholar  

Nasreddine Z, Phillips N, Bédirian V, Charbonneau S, Whitehead V, Collin I, Cummings J, Chertkow H. The montreal cognitive assessment, MoCA: A brief screening tool for mild cognitive impairment. JAGS. 2005;53:695–9.

Morris JC, Heyman A, Mohs RC, Hughes JP, van Belle G, Fillenbaum G, Mellits ED, Clark C. The Consortium to Establish a Registry for Alzheimer’s Disease (CERAD). Part I. Clinical and neuropsychological assessment of Alzheimer’s disease. Neurology. 1989;39(9):1159–65.

Article   CAS   PubMed   Google Scholar  

Norwegian Registry for Primary Health Care [ https://helsedata.no/en/forvaltere/norwegian-directorate-of-health/norwegian-registry-for-primary-health-care-kpr/ ]. Accessed 12 May 2023.

Bakken IJ, Ariansen AMS, Knudsen GP, Johansen KI, Vollset SE. The Norwegian Patient Registry and the Norwegian Registry for Primary Health Care: Research potential of two nationwide health-care registries. Scand J Public Health. 2020;48(1):49–55.

Norwegian Patient Registry [ https://helsedata.no/en/forvaltere/norwegian-directorate-of-health/norwegian-patient-registry-npr/ ]. Accessed 12 May 2023.

StataCorp. Stata Statistical Software: Release 16. College Station: StataCorp LLC; 2019.

Zhang X, Dupre ME, Qiu L, Zhou W, Zhao Y, Gu D. Age and sex differences in the association between access to medical care and health outcomes among older Chinese. BMC Health Serv Res. 2018;18(1):1004.

Bale TL, Epperson CN. Sex as a Biological Variable: Who, What, When, Why, and How. Neuropsychopharmacology. 2017;42(2):386–96.

Johnsen TM, Norberg BL, Kristiansen E, Zanaboni P, Austad B, Krogh FH, Getz L. Suitability of Video Consultations During the COVID-19 Pandemic Lockdown: Cross-sectional Survey Among Norwegian General Practitioners. J Med Internet Res. 2021;23(2):e26433.

Bell SA, Krienke L, Brown A, Inloes J, Rettell Z, Wyte-Lake T. Barriers and facilitators to providing home-based care in a pandemic: policy and practice implications. BMC Geriatr. 2022;22(1):234.

Rokstad AMM, Røsvik J, Fossberg M, Eriksen S. The COVID-19 pandemic as experienced by the spouses of home-dwelling people with dementia - a qualitative study. BMC Geriatr. 2021;21(1):583.

Tuijt R, Frost R, Wilcock J, Robinson L, Manthorpe J, Rait G, Walters K. Life under lockdown and social restrictions - the experiences of people living with dementia and their carers during the COVID-19 pandemic in England. BMC Geriatr. 2021;21(1):301.

Morina N, Kip A, Hoppen TH, Priebe S, Meyer T. Potential impact of physical distancing on physical and mental health: a rapid narrative umbrella review of meta-analyses on the link between social connection and health. BMJ Open. 2021;11(3):e042335.

Greenblatt-Kimron L, Shinan-Altman S, Alperin M, Levkovich I. Depression and Medicine Use among Older Adults during the COVID-19 Pandemic: The Role of Psychosocial Resources and COVID-19 Perceived Susceptibility. Int J Environ Res Public Health. 2023;20(4):3398.

Vigezzi GP, Bertuccio P, Amerio A, Bosetti C, Gori D, Cavalieri d’Oro L, Iacoviello L, Stuckler D, Zucchi A, Gallus S, et al. Older Adults’ Access to Care during the COVID-19 Pandemic: Results from the LOckdown and LifeSTyles (LOST) in Lombardia Project. Int J Environ Res Public Health. 2022;19(18):11271.

Helgeland J, Telle KE, Grøsland M, Huseby BM, Håberg S, Lindman ASE. Admissions to Norwegian Hospitals during the COVID-19 Pandemic. Scand J Public Health. 2021;49(7):681–8.

Smith M, Vaughan Sarrazin M, Wang X, Nordby P, Yu M, DeLonay AJ, Jaffery J. Risk from delayed or missed care and non-COVID-19 outcomes for older patients with chronic conditions during the pandemic. J Am Geriatr Soc. 2022;70(5):1314–24.

World Health Organization. Third round of the global pulse survey on continuity of essential health services during the COVID-19 pandemic: November–December 2021. In. Geneva: World Health Organization; 2022.

Norwegian Institute of Public Health: Influensasesongen i Norge 2019–2020 [Influenza season in Norway 2019–2020] [ https://www.fhi.no/publ/2020/influensasesongen-i-norge-2019-2020/ ]. Accessed 22 June 2023.

Norwegian Institute of Public Health: Influensasesongen i Norge 2021–2022 [Influenza season in Norway 2021–2022] [ https://www.fhi.no/publ/2022/influensasesongen-i-norge-2021-2022/ ]. Accessed 22 June 2023.

Oh KB, Doherty TM, Vetter V, Bonanni P. Lifting non-pharmaceutical interventions following the COVID-19 pandemic - the quiet before the storm? Expert Rev Vaccines. 2022;21(11):1541–53.

Greig F, Perera G, Tsamakis K, Stewart R, Velayudhan L, Mueller C. Loneliness in older adult mental health services during the COVID-19 pandemic and before: Associations with disability, functioning and pharmacotherapy. Int J Geriatr Psychiatry. 2021;37(1):10.

PubMed Central   Google Scholar  

Download references

Acknowledgements

HUNT is a collaborative project between the HUNT Research Centre at the Faculty of Medicine and Health Sciences, Norwegian University of Science and Technology, the Trøndelag County Council, the Central Norway Regional Health Authority and the Norwegian Institute of Public Health. We would like to thank everyone who participated in HUNT 70+ for their valuable contributions to this research.

This study was supported by the Norwegian Health Association (grant no. 22687).

Author information

Authors and affiliations.

The Norwegian National Centre for Ageing and Health (Ageing and Health), Vestfold Hospital Trust, Tønsberg, Norway

Tanja Louise Ibsen, Bjørn Heine Strand, Sverre Bergh, Anne Marie Mork Rokstad & Geir Selbæk

Department of Geriatric Medicine, Oslo University Hospital, Oslo, Norway

Bjørn Heine Strand

Department of Physical Health and Ageing, Norwegian Institute of Public Health, Oslo, Norway

Research Centre for Age-Related Functional Decline and Disease (AFS), Innlandet Hospital Trust, Ottestad, Norway

Sverre Bergh

Division of Psychiatry, University College London, London, UK

Gill Livingston

Camden and Islington NHS Foundation Trust, London, UK

Health Services Research Unit, Akershus University Hospital, Oslo, Norway

Hilde Lurås

Institute of Clinical Medicine, University of Oslo, Oslo, Norway

Centre for Research On Pandemics & Society (PANSOC), at Oslo Metropolitan University, Oslo, Norway

Svenn-Erik Mamelund

Department of Psychiatry, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands

Richard Oude Voshaar

Faculty of Health Sciences and Social Care, Molde University College, Molde, Norway

Anne Marie Mork Rokstad

Department of Neuromedicine and Movement Science, Faculty of Medicine and Health Science, Norwegian University of Science and Technology, Trondheim, Norway

Pernille Thingstad

Department of Health and Social Services, Trondheim Municipality, Trondheim, Norway

Department of Primary and Community Care, Research Institute for Medical Innovation, Radboudumc Alzheimer Center, Radboud University Medical Center, Nijmegen, Netherlands

Debby Gerritsen

Geir Selbæk

Institute of Clinical Medicine, Faculty of Medicine, University of Oslo, Oslo, Norway

You can also search for this author in PubMed   Google Scholar

Contributions

GS led the study project and is responsible for the concept and design of the study, together with BHS, SB and TLI. BHS was a major contributor in the analysis prosses together with TLI. TLI, BHS, SB, GL, HL, SEM, ROV, AMMR, PT og GS contributed to interpreting the data. TLI drafted the paper, with substantially contributions from all the authors in revising the drafted work. DG made significant contributions on the revised version after peer review. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Tanja Louise Ibsen .

Ethics declarations

Ethics approval and consent to particpate.

This study was approved by the Regional Committee for Medical and Health Research Ethics of Norway (REK Southeast B 182575). All methods were carried out in accordance with REK’s guidelines which correspond to the Declaration of Helsinki. The present study is part of a larger project registered at ClinicalTrials.gov (identification number: NCT 04792086). Informed written consent was obtained from all participants in the HUNT4 70 + study. Participants with reduced capacity to consent were included if they had a next of kin who consented on their behalf. In the consent form, it was thoroughly described that collected data can be linked to other registers in order to carry out approved research projects, as has been done in the present project.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Ibsen, T.L., Strand, B.H., Bergh, S. et al. A longitudinal cohort study on the use of health and care services by older adults living at home with/without dementia before and during the COVID-19 pandemic: the HUNT study. BMC Health Serv Res 24 , 485 (2024). https://doi.org/10.1186/s12913-024-10846-y

Download citation

Received : 07 July 2023

Accepted : 11 March 2024

Published : 19 April 2024

DOI : https://doi.org/10.1186/s12913-024-10846-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health care services
• Older adults
• Longitudinal cohort study

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

This paper is in the following e-collection/theme issue:

Published on 30.4.2024 in Vol 8 (2024)

Precision Assessment of Real-World Associations Between Stress and Sleep Duration Using Actigraphy Data Collected Continuously for an Academic Year: Individual-Level Modeling Study

Authors of this article:

Original Paper

• Constanza M Vidal Bustamante 1, 2 * , PhD   ; 
• Garth Coombs III 1, 2 * , PhD   ; 
• Habiballah Rahimi-Eichi 1, 2, 3, 4 , PhD   ; 
• Patrick Mair 1 , PhD   ; 
• Jukka-Pekka Onnela 5 , PhD   ; 
• Justin T Baker 3, 4 , PhD, MD   ; 
• Randy L Buckner 1, 2, 4, 6 , PhD  

1 Department of Psychology, Harvard University, Cambridge, MA, United States

2 Center for Brain Science, Harvard University, Cambridge, MA, United States

3 Institute for Technology in Psychiatry, McLean Hospital, Belmont, MA, United States

4 Department of Psychiatry, Harvard Medical School, Boston, MA, United States

5 Department of Biostatistics, Harvard University, Boston, MA, United States

6 Athinoula A Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, United States

*these authors contributed equally

Corresponding Author:

Constanza M Vidal Bustamante, PhD

Department of Psychology

Harvard University

52 Oxford Street

Northwest Building, East Wing, Room 295.06

Cambridge, MA, 02138

United States

Phone: 1 617 384 8230

Email: [email protected]

Background: Heightened stress and insufficient sleep are common in the transition to college, often co-occur, and have both been linked to negative health outcomes. A challenge concerns disentangling whether perceived stress precedes or succeeds changes in sleep. These day-to-day associations may vary across individuals, but short study periods and group-level analyses in prior research may have obscured person-specific phenotypes.

Objective: This study aims to obtain stable estimates of lead-lag associations between perceived stress and objective sleep duration in the individual, unbiased by the group, by developing an individual-level linear model that can leverage intensive longitudinal data while remaining parsimonious.

Methods: In total, 55 college students (n=6, 11% second-year students and n=49, 89% first-year students) volunteered to provide daily self-reports of perceived stress via a smartphone app and wore an actigraphy wristband for the estimation of daily sleep duration continuously throughout the academic year (median usable daily observations per participant: 178, IQR 65.5). The individual-level linear model, developed in a Bayesian framework, included the predictor and outcome of interest and a covariate for the day of the week to account for weekly patterns. We validated the model on the cohort of second-year students (n=6, used as a pilot sample) by applying it to variables expected to correlate positively within individuals: objective sleep duration and self-reported sleep quality. The model was then applied to the fully independent target sample of first-year students (n=49) for the examination of bidirectional associations between daily stress levels and sleep duration.

Results: Proof-of-concept analyses captured expected associations between objective sleep duration and subjective sleep quality in every pilot participant. Target analyses revealed negative associations between sleep duration and perceived stress in most of the participants (45/49, 92%), but their temporal association varied. Of the 49 participants, 19 (39%) showed a significant association (probability of direction>0.975): 8 (16%) showed elevated stress in the day associated with shorter sleep later that night, 5 (10%) showed shorter sleep associated with elevated stress the next day, and 6 (12%) showed both directions of association. Of note, when analyzed using a group-based multilevel model, individual estimates were systematically attenuated, and some even reversed sign.

Conclusions: The dynamic interplay of stress and sleep in daily life is likely person specific. Paired with intensive longitudinal data, our individual-level linear model provides a precision framework for the estimation of stable real-world behavioral and psychological dynamics and may support the personalized prioritization of intervention targets for health and well-being.

Introduction

The transition to college is often accompanied by elevated stress and insufficient sleep [ 1 - 3 ], both of which have been found to impact daily functioning and, upon repeated exposure, to be associated with negative health outcomes ranging from internalizing disorders, including anxiety and depression, to cardiac and metabolic disease [ 4 - 9 ]. A better understanding of the dynamic day-to-day interplay between perceived stress and sleep duration could inform the mechanisms of their downstream impacts and the development of interventions to prevent or mitigate them [ 10 - 14 ]. Nevertheless, only a few studies have collected daily observations to evaluate within-person associations between stress responses and sleep duration, and their results are mixed: some studies report that heightened stress is followed by shorter sleep that night but not vice versa [ 15 , 16 ], others report that shorter sleep is followed by heightened stress the next day but not vice versa [ 17 ], and yet others report bidirectional relationships [ 18 , 19 ].

Inconsistent findings in stress-sleep associations might be at least partially explained by 2 interrelated methodological limitations, both of which are addressed by the individual-level modeling approach presented in this study. First, existing longitudinal studies have aggregated individual observations collected over short study periods of ≤14 days, thus limiting the estimation of stable associations that are robust to changing environmental demands (eg, first week of the semester vs final examinations period). Thanks to the adoption of digital phenotyping tools such as wearables and smartphones, research designs that sample individuals over much longer periods of time are increasingly feasible [ 20 - 22 ]. Sleep duration can now be passively tracked through continuous actigraphy sensing over months and even years, while perceived stress levels can be probed via brief daily smartphone-based surveys, with high compliance rates in student samples and small participant burden [ 23 - 25 ].

Moreover, the prevailing focus on group modeling and sample-level effects obscures the possibility that day-to-day associations between perceived stress and sleep duration may vary across individuals; for instance, it is possible that in certain individuals, getting less sleep than usual has no significant impact on stress levels the following day, but heightened stress during the day leads to shorter sleep duration later that night, while the opposite pattern might be true for others. Even when group-level models allow for individual-level estimates, such as in multilevel models with random effects, the degree to which the individual estimates are pulled toward the group (a shrinkage effect) is different for each individual based on the amount of data they provide, thus reducing the individual tailoring in a nonuniform manner [ 26 ].

Individual-level linear models (iLMs), which are fitted over a single individual’s intensive longitudinal observations, may offer a critical alternative to the estimation of stress-sleep associations. Compared to group-level approaches, iLMs provide estimates of phenotypes that are tailored to each person’s data and unbiased by the group [ 27 - 30 ]. They may also be more readily applicable in real-world contexts, where a clinician might use a precision health approach to evaluate and support each individual patient based on their personal data rather than on a hypothetical average patient [ 31 - 34 ]. Of note, individual-level approaches can also contribute to conclusions at the group or population level, such as by estimating the prevalence of each individually derived phenotype, rather than blurring across individuals, to arrive at a central tendency that might not accurately represent many of the included individuals.

In this study, we introduce a novel iLM approach that leverages daily observations collected over a full academic year for the assessment of day-to-day associations between actigraphy-derived sleep duration and self-reported stress levels in first-year college students. Our aims were 2-fold. First, we used a pilot data set to develop and validate a parsimonious iLM that estimates concurrent or lagged associations between 2 daily variables of interest while accounting for the weekly structure in student behavior. We then leveraged this iLM for the target examination of bidirectional day-to-day associations between perceived stress and objective sleep duration in an independent data set of 49 first-year college students. We expected a negative association for most participants—such that higher stress levels are associated with shorter sleep—but we also anticipated that some participants might show positive or null associations. We further hypothesized that the lead-lag relationship between perceived stress and sleep duration (ie, which of the 2 temporally precedes the other) would vary across individuals such that, for some, elevated stress would associate with shorter sleep that night; for others, shorter sleep would associate with elevated stress the next day; and for still others, both directions of association would be identified.

Participants

Pilot group.

A total of 6 undergraduate students returning for their sophomore year volunteered for a year-long study (all aged 19 years; n=3, 50% women and n=3, 50% men). All had participated in a previous pilot study in our laboratory during their first year of college and were known to be compliant. These 6 pilot participants provided data to develop the statistical models that were then applied to the new group of target participants described in the next subsection. The pilot participants enrolled for this study during the first 2 weeks of their fall semester. As in the case of the target participants, they were required to be taking full-time classes and own a smartphone compatible with the study smartphone app, Beiwe, which is part of the open-source Beiwe platform for digital phenotyping [ 35 ]. Available and missing data information at the participant level is provided in Figure S1 in Multimedia Appendix 1 . Given our focus on individual-level models where each person serves as their own baseline, and there is no aggregation across participants, students were not excluded for current or past psychiatric disorders or medication use, and nor were they excluded if they began treatment or medication for mental health issues during the study.

Target Group

In total, 49 undergraduate students beginning their first year of college volunteered for a year-long study (aged 18-19 years; mean age 18.1, SD 0.24 y; n=25, 51% women and n=24, 49% men). We have previously reported data from these participants [ 24 ]. Participants living on campus were recruited via flyers posted on campus boards and distributed via email lists and were enrolled during the first 2 weeks concurrently with the pilot participants. Enrollment criteria were the same as for the pilot participants other than the fact that the target first-year participants were all new to the university. Initially, 68 participants enrolled, of whom 19 (28%) were excluded based on issues with data acquisition, including early withdrawal from the study (n=7, 37%), technical failure of the actigraphy data (n=1, 5%), poor-quality actigraphy data (n=2, 11%), and completion of <50% of the daily surveys (n=9, 47%). Of the 49 participants in the final sample, 2 (4%) identified as American Indian, 5 (10%) as Asian, 7 (14%) as Black, 31 (63%) as White, and 2 (4%) as mixed race. Furthermore, 12% (6/49) reported prior diagnosis of a psychiatric disorder (including anxiety, depression, and attention-deficit/hyperactivity disorder), and 8% (4/49) maintained active diagnoses. The 49 first-year students had not yet declared their area of study, but they reported their desired future occupation to be medicine (n=15, 31%), business or finance (n=7, 14%), academia or other research (n=6, 12%), engineering (n=5, 10%), policy or government (n=5, 10%), law (n=4, 8%), and other or undecided (n=7, 14%). Of the 49 participants, 46 (94%) were iPhone users, whereas 3 (6%) used Android mobile phones.

Study Design

As previously reported in our study [ 24 ], this intensive longitudinal observational study collected passive and active data as participants engaged in their lives over a full academic year, extending a few days into the summer break. Both pilot and target participants completed smartphone-based daily surveys and a voice-recorded diary; wore an actigraphy wristband (GENEActiv Original; Activinsights Ltd) for continuous activity and sleep monitoring for the duration of the study; completed a battery of web-based questionnaires at the beginning, middle, and end of the study; and attended brief in-person check-ins every 3 to 4 weeks.

Ethical Considerations

Informed consent and all study procedures and methods were approved by the institutional review board of Harvard University (IRB16-1230). All participants completed an in-person informed consent session where study procedures were explained, and any questions were clarified. Participants were informed that they could withdraw their study participation at any time. Participants were compensated US $1 per each daily survey they submitted, US $1 per day for continuously wearing the actigraphy wristband, and US $20 per hour for web-based surveys and attending in-person visits. Milestone bonus payments for completing half of the study (US $100) and the full study (US $300) were also provided to compensate participants for their continued compliance.

Study data collected across devices were stored and automatically backed up in a secure data warehouse configured to automatically import data from various collection streams. All data were kept as securely as possible and were only accessible to study staff. Participants’ data were labeled with a randomly generated participant ID. Personally identifying information, such as names and contact information, were kept separate from all other collected data in a locked file cabinet (in a locked office, behind an ID card–secured suite during off-hours) and in a password-protected database.

Measures and Quality Control

Objective sleep duration.

Daily sleep duration reflects the number of minutes between the estimated start and end of the day’s longest detected sleep episode. As redescribed from our original study [ 24 ], sleep duration was derived from the accelerometer data collected through the GENEActiv Original actigraphy wristband and analyzed via the deep phenotyping of sleep processing pipeline [ 36 ]. Participants wore the wristband on their nondominant wrist continuously, including during sleep and when bathing. Triaxial acceleration was collected with a sampling frequency of 30 Hz during the academic semesters and 10 Hz during the winter break (to extend battery life and memory while participants were away from campus). Participants were instructed to press the wristband’s button when they began trying to fall sleep at night and immediately after they awoke in the morning. Individuals exchanged their wristband for a fully charged one with reset memory at the in-person check-ins.

The deep phenotyping of sleep (DPSleep) processing pipeline was applied to the raw actigraphy data to detect the major sleep episode for each day [ 36 ]. The pipeline first converted the accelerometer data to minute-based activity estimates, removed the minutes when the individual was not wearing the device, and then estimated the major sleep episode based on a sliding window. Days where one of the boundaries of the sleep episode (ie, rises in relative activity both before and after a period of lower activity) could not be detected due to missing data were labeled as unusable. Two independent trained raters examined the automatically detected start and end times and the usability label of each sleep episode against the minute-based activity levels and the participant button presses when available. When necessary, they adjusted the automatic times and labels. Any disagreements between the 2 raters’ assessments were reviewed by the research team and resolved through discussion. A full description of the DPSleep processing pipelines applied to the actigraphy data, including quality control steps, can be found in the study by Rahimi-Eichi et al [ 36 ].

All data that passed quality control were included in analysis, including days with no detected sleep episode (ie, with no extended periods of lower relative activity).

Daily Telephone-Based Surveys

Smartphone surveys were administered via the Beiwe app [ 35 ]. Each night before bed, participants completed a 46-item self-report survey related to their daily lives. As described originally in our study [ 24 ], the questions assessed a range of behaviors and internal states over the past 24 hours, including sleep quality, stress levels and sources, positive and negative affect, general physical health, daily consumption habits, studying behaviors, and sociability and support [ 24 ]. This paper reports analyses using 2 survey questions selected a priori that probed daily subjective sleep quality and perceived stress. The sleep quality question asked “How did you sleep last night?” and was answered on a 5-point scale (1=terribly: little or no sleep, 2=not so well: got some sleep but not enough, 3=sufficient: got enough sleep to function, 4=good: got a solid night’s sleep and felt well rested, and 5=exceptional: one of my best nights of sleep). The perceived stress question asked “How much did you feel stressed over the past 24 hours?” and was also answered on a 5-point scale (1=very slightly or not at all, 2=a little, 3=moderately, 4=quite a bit, and 5=extremely).

Surveys submitted between 5 PM (local time) on the day the survey opened and 6 AM the following day were considered to be on time. Surveys submitted after 6 AM the day after the survey was prompted were marked as missing. A participant was included in analysis if they were compliant with at least 100 daily surveys across the data collection period, and only on-time surveys from these participants were included.

Analytical Approach

Development of the ilm.

An iLM was developed on the intensive day-level longitudinal data from the 6 pilot participants to test person-specific day-to-day behavioral associations. The iLM framework allows for individually tailored estimates by treating each day as the unit of observation and the individual as the population , as opposed to each individual as the unit of observation used to estimate the general population. An individual’s observed time series data can be understood as just one realization of a stochastic process whose data-generating process we are trying to model and understand [ 37 ], in this case through linear models. The intercept and slope estimates are unique to the individual, and we interpret them as a proxy for the individual’s phenotype. Of note, in this framework, each individual model can be interpreted as an independent test of the hypothesized association (eg, is shorter sleep than usual associated with heightened perceived stress the following day?).

The model was structured to be parsimonious while accounting for the nonindependence of the daily measures and the temporal structure imposed by the academic schedule. The day of the week was included as a categorical covariate to account for weekend versus weekday effects and other weekly structures imposed by the college schedule (eg, classes that meet on Monday-Wednesday-Friday vs those that meet on Tuesday-Thursday). In addition, because behavioral patterns during the semester vary substantially from those during the 5-week winter break (when students do not have classes and typically are away from campus), we decided a priori to only include in the model compliant data collected during the school semesters.

The final iLM took the following general form:

y t = β0 + β1x t or t−1 + β2DayOfWeek t + ε t

A daily observation on day t starts with the nighttime sleep episode and ends with the submission of the daily survey submitted in the evening before the next sleep episode. In the aforementioned formula, y t is a single participant’s outcome variable (eg, sleep duration in min) at daily time point t , β 0 is this participant’s individual intercept, x is the predictor variable (eg, sleep quality or stress measured on a 1-5 scale) at daily time point t or t−1 (depending on the lag of the tested association), β 1 is the participant’s individual slope for x , DayOfWeek t is the day of the week in which the outcome observation was acquired (modeled as a categorical variable ordered from Saturday to Friday), and ε t is a normally distributed random error term.

All modeling was carried out in a Bayesian inference framework, which treats unknown parameters (eg, a slope) as random variables with a probability distribution rather than a discrete value; this distribution is updated based on the observed data (resulting in a posterior distribution ) and serves as a measure of uncertainty around the parameter [ 38 - 40 ]. The Bayesian framework was favored for these analyses due to its flexibility in computing models with varied specifications (including complex random effect specifications in the multilevel models we fitted as part of our model validation process), robustness to sample data characteristics (eg, dispersion), and intuitive interpretation of the posterior distribution and 95% uncertainty interval (UI; ie, conditional on the data and the model, the probability that the parameter is contained in the interval is 0.95) [ 39 , 40 ]. For comparison, parallel iLMs fitted in a frequentist inference framework in the pilot validation stage yielded nearly identical point estimates to their Bayesian counterparts (refer to Figure S2 in Multimedia Appendix 1 ), suggesting that our model specification is robust across both statistical inference frameworks. All models were estimated in R (version 4.3.1; R Foundation for Statistical Computing) [ 41 ]. Bayesian models were estimated using the Stan modeling language [ 42 ] and the packages rstanarm (version 2.21.4 [ 43 ]), tidybayes (version 3.0.1 [ 44 ]), and bayestestR (version 0.13.1 [ 45 ]). Frequentist iLMs were estimated using the stats package included in base R [ 41 ].

Bayesian models were fit with default weakly informative priors specifying a gaussian distribution (mean 0, SD 2.5) to represent our diffuse prior knowledge. We estimated parameters using a Markov chain Monte Carlo (MCMC) approach. For each parameter, we sampled from 4 stationary Markov chains, each comprising 5000 sampling iterations, including a burn-in period of 2500 iterations that were discarded (for a total of 10,000 post–warm-up draws). Convergence of the 4 chains to a single stationary distribution was assessed quantitatively via the R-hat convergence diagnostic [ 46 ] (adequate convergence defined as R-hat <1.1) and qualitatively by visual inspection of trace plots showing the estimated parameter as a function of each chain’s iteration number (adequate convergence defined as the chains overlapping with each other throughout and a lack of structured patterns in each chain). Each model’s effective sample size (ESS) metric is reported. Each estimate in the MCMC process is serially correlated with the previous estimates: the higher the correlation, the more samples are needed to get to a stationary distribution. In the presence of nearly no autocorrelation, the ESS will be equal to the number of posterior draws requested (in this case, 10,000). Generally, the ESS should be at least 1000 to obtain stable estimates [ 38 , 47 ].

Adequacy of the model specification was assessed via 2 methods. First, posterior predictive checks entailed a visual comparison of the distribution of the observed outcome variable to the distribution of 100 simulated outcome data sets generated by applying 100 draws from the model parameters’ posterior distribution to our observed data set. Similarity in the distributions of the observed and model-generated outcomes suggests that the model specification captured the data well. Second, we inspected the model residuals against the model’s predicted values to confirm homoscedasticity and against themselves to rule out problematic autocorrelation due to temporal dependencies in longitudinal data.

Point estimates of intercepts and slopes were computed as the median value of their respective posterior distributions. Furthermore, 95% UIs were computed as the 2.5% and 97.5% quantiles of the posterior distribution. To provide intuitive parallels to a frequentist inference framework, we interpreted a predictor slope as statistically significant if its 95% UI did not contain 0, or, put differently, if the proportion of the posterior distribution falling in the direction (positive or negative) of the point estimate (also known as probability of direction [pd]) was higher than 0.975 (which approximates a frequentist 1-tailed P value of <.025) [ 48 ]. Although we focus on 95% as the cutoff for the UI given the widespread use of this number in the literature, it should be noted that this threshold has been criticized as arbitrary [ 39 ]. To go beyond testing whether the slope is different from exactly 0, we also report the percentage of the 95% UI that falls within a region of practical equivalence (ROPE), defined as parameter values that are sufficiently close to 0 to be considered equivalent to the null for practical purposes [ 49 ] and mathematically defined as a standardized effect size of <0.1 (ie, half of a small effect as defined by Cohen [ 50 ]). Given our individualized approach, ROPEs were estimated separately for each participant based on their observed data.

Validation of the iLM in the Pilot Data

Data from 6 pilot participants were used to test the properties of the iLM and explore associations among variables of interest to validate the approach. A first sanity check was to assess whether the iLM captured the expected association between a sleep event’s objective duration (estimated from an actigraphy wristband) and the subjective rating (reported at the end of each day as part of a smartphone-based survey) for the same ( concurrent ) sleep event. Participants were expected to rate episodes of shortened sleep as worse quality compared to nights of longer sleep. This contemporaneous model, known as a static model in the time series literature [ 37 ], was specified as follows:

ObjectiveSleepDuration t = β 0 + β 1 SubjectiveSleepRating t + β 2 DayOfWeek t + ε t

A second, exploratory lagged model was fit testing associations between sleep duration on day t and sleep quality reported the day before, on day t−1 . We expected that this lagged association would be weaker than the concurrent association outlined previously (given that the variables tested are no longer referring to the same sleep event), and importantly, that the association would be in the opposite direction such that poor-quality sleep on one night is associated with longer sleep duration the next night, signaling a compensatory sleep rebound effect. To test this, a model similar to the aforementioned one was fit, with the difference that the predictor was lagged by 1 day. This lagged model, known as a finite distributed lag model in time series analysis [ 37 ], was specified as follows:

ObjectiveSleepDuration t = β 0 + β 1 SubjectiveSleepRating t−1 + β 2 DayOfWeek t + ε t

Both models were fit with a gaussian distribution to reflect the observed normal distribution of sleep duration in our sample.

We conducted model checks to evaluate the performance of the iLM framework. Posterior predictive checks (described earlier in this subsection) evaluated that a gaussian model specification captured the distribution of the data well. In addition, given the longitudinal design, we examined whether the model residuals lacked meaningful autocorrelation, which would suggest that our day-of-the-week covariate sufficiently captured the weekly structure of sleep duration. Finally, we compared the estimates obtained through the iLMs to the estimates obtained when the same data for the 6 pilot participants were fit within a single, group-based multilevel linear model (MLM), specifying fixed effects for intercepts and slopes and additional participant-level random intercepts and random slopes for the main predictor of interest (in this case, sleep quality). This comparison allowed us to assess our expectation that MLMs would provide individual estimates roughly comparable to those of the iLM, but with the critical difference that MLMs would systematically attenuate these individual estimates, especially for individuals who deviate from the predominant association phenotype or when there is large heterogeneity in these phenotypes across individuals.

Application of the iLM to the Novel Target Participants

After developing the iLM and validating it extensively over the pilot data set, the modeling framework was then carried forward and applied to the independent target data set of first-year college students. All models were fit with a gaussian distribution. We first applied the same sanity check models as in the pilot data set. Subsequently, we used the iLM method to test a priori target hypotheses regarding the association between perceived daily stress and objective sleep duration.

Two target models were fit for each individual to examine bidirectional associations between sleep duration and perceived stress. A daily observation on day t consists of last night’s sleep duration ( ObjectiveSleepDuration t , recorded passively via the actigraphy wristband) and the subjective rating of the present day’s overall stress levels ( SubjectiveStressRating t, reported by the participant in the evening). First, to test whether stress level during the day is related to sleep duration later that night (ie, a stress-then-sleep association), the model used the stress rating the day before sleep (day t−1 ) as the predictor of subsequent sleep duration (day t ). Thus, the formula for this model was specified as follows:

ObjectiveSleepDuration t = β 0 + β 1 SubjectiveStressRating t−1 + β 2 DayOfWeek t + ε t

Next, to test whether sleep duration is related to stress levels the day after (ie, a sleep-then-stress association), the model used stress rating the day after sleep (day t ) as predictor of the previous sleep duration (day t , sleep duration last night):

ObjectiveSleepDuration t = β 0 + β 1 SubjectiveStressRating t + β 2 DayOfWeek t + ε t

In this model, stress rating (predictor) is back-predicting sleep duration (outcome), which effectively tests the question “Is stress today associated with sleep last night?” As the objective sleep duration measure occurs temporally before the daily stress rating is submitted, results are interpreted as sleep duration being associated with increased or decreased stress the next day. Implementing the model in this way (rather than using sleep duration as the predictor and stress rating as the outcome) presented the advantage that the slope estimates across the 2 models are on equivalent units, namely, the change in the number of minutes in sleep duration per unit of change in stress rating. This allows us to directly compare the quantitative outputs for the models testing the questions “Is stress associated with subsequent sleep?” and “Is sleep associated with subsequent stress?”

As with the pilot data set, we conducted model checks to further evaluate the specification and performance of the iLM framework in our target data set, including posterior predictive checks; inspection of model residuals; and a comparison of the estimates obtained through the iLMs and the estimates obtained when the same data for the target sample were fit in a single, group-level MLM with random intercepts and random slopes per participant on the stress predictor. Additional inspection of model results against each participant’s raw time series data was conducted to complement our interpretation.

Participant-Level Descriptive Statistics

Table 1 presents participant-level available data and summary statistics for sleep and stress variables used in analysis. The pilot participants provided a median of 178 (range 119-212 out of a total possible of 223) usable observations for modeling, that is, day-level observations collected during the school semesters with usable actigraphy and survey data. The target participants provided a median of 178 (range 84-214) usable observations. Participants’ total number of usable observations was not correlated with their mean sleep duration ( r =0.09; P =.53) or mean stress levels ( r =−0.11; P =.42). In addition, participants’ mean sleep duration did not differ on days with or without available survey data (paired 2-tailed t test, P =.59), and participants’ mean stress levels did not differ on days with or without available actigraphy data (paired 2-tailed t test, P =.70). These observations suggest that participants’ overall sleep and stress metrics did not introduce systematic missingness in the data (more details on participant-level missing data are presented in Figure S1 in Multimedia Appendix 1 ).

a IDs starting with “P” indicate pilot sample participants.

b IDs starting with “T” indicate target sample participants.

Sleep and Stress Fluctuate in Relation to the Academic Calendar in the Pilot Participants

There was a pattern of enhanced sleep and lower stress when students were released from the structured academic demands of the in-person school semesters. Participants were enrolled for a full academic year, including a fall semester and a spring semester (each lasting roughly 16 weeks), as well as a 5-week class-free winter break bridging the 2 semesters. During the winter break and weekends, pilot participants had longer objective (actigraphy-derived) sleep duration, better subjective sleep quality, and felt less stress compared to school semesters and weekdays ( Figures 1 A and 1B). The temporal structure of sleep and stress variables was further evidenced by their autocorrelation estimates. Autocorrelations were generally small (|r|<0.2; Figure 1 C) but were strongest (highlighted with asterisks) at a 7-day lag (and again at a 14-day lag) for sleep duration, consistent with a weekly sleep schedule. By contrast, autocorrelations for stress were strongest at 1- and 2-day lags, suggesting that experiences of stress might come in chains of >1 day.

The school break and week-related changes observed in the data informed the design of the iLM seeking to capture stable person-level associations. Given that the winter break presents different environmental demands from the school semester, we decided a priori to exclude from the model the observations collected during this period. In addition, to account for weekly patterns in sleep behavior (outcome variable) within the school semesters, we added the day of the week as a covariate in the model.

iLMs Capture Day-to-Day Associations in the Pilot Participants

To test the viability of the iLM approach, a proof-of-concept model examined the association between objective sleep duration and subjective sleep quality in the pilot participants ( Table 2 and Figure 2 ). This model allowed for a test of construct validity, given that the tested association was intuitive and expected. The model tested the association between sleep duration and sleep quality for the same sleep episode (ie, a concurrent association). All pilot participants showed the expected positive association: when participants slept for a shorter period than usual, they also subjectively rated these same sleep events as worse quality (shown in orange in Figure 2 A). The effect size was large for all individuals, and the 95% UI lay outside of the ROPE for all models. These results provided preliminary evidence that the iLM is a valid framework to reliably detect relations between psychological variables (in this case, subjective sleep quality rating) and objective behaviors (sleep duration) at the individual participant level.

A second, exploratory iLM tested the association between sleep duration (on day t ) and sleep quality the day before (on day t−1 ). We expected this model to show significant but weaker effects compared to the first model, given that the variables were referring to lagged sleep events. Moreover, we expected negative associations such that worse sleep quality on one night would associate with longer sleep duration the next night, that is, a compensatory sleep rebound effect. Half of the participants (3/6, 50%; P1, P4, and P6) showed this expected pattern of association (pd>0.975; shown in blue in Figure 2 ). Of the 6 participants, 1 (17%; P2) showed a positive association such that worse subjective sleep quality experienced on the preceding night was associated with shorter sleep durations on the subsequent night. This participant might experience chains of poor sleep over multiple days (eg, reduced sleep days ahead of a deadline to accommodate increased workload), rather than a sleep rebound effect immediately the following night. In addition, of the 6 participants, 2 (33%; P3 and P5) showed a positive slope estimate but without reaching statistical significance (pd<0.975). We did not observe structured patterns of association between individual slope estimates and individuals’ mean sleep quality, mean sleep duration, or total number of daily observations ( Figure 2 B). This suggests that the estimated slopes were not systematically influenced by person-level characteristics of the psychological and behavioral phenomena of interest.

a UI: uncertainty interval.

b pd: probability of direction.

c ROPE: region of practical equivalence.

d ESS: effective sample size.

e Statistically significant result.

Model Diagnostics Confirm Convergence and Adequate Specification

Both visual and quantitative MCMC diagnostic checks revealed that all iLMs converged successfully ( Figure 3 ). Trace plots ( Figure 3 , column 1) revealed no structured pattern in the estimated slopes across sampling iterations. R-hat values were <1.1, and ESSs were >1000 for the estimated slopes of all models ( Table 2 ). Posterior predictive checks comparing the observed distribution of the outcome variable (sleep duration) to 100 randomly sampled simulated data sets from the posterior predictive distribution confirmed that a gaussian model specification captured the observed data well ( Figure 3 , column 2). Inspection of the model residuals against the model’s predicted values confirmed homoscedasticity, with no structured pattern in the plots ( Figure 3 , columns 3 and 4). Finally, residual autocorrelation was generally low (| r |<0.2) for all models ( Figure 3 , column 5), suggesting that the model specification was able to account for the temporal structure in the data.

iLMs Yield Similar, but Not Identical, Estimates to a Group-Based Multilevel Model

The slopes estimated from the iLM are similar to those obtained through a MLM testing the same associations between concurrent objective sleep duration and subjective sleep quality ( Figure 4 ). This suggests that the individually tailored slopes obtained from our iLM are comparable to more traditional group-based approaches. Of note, although the estimates are similar, they are not identical.

Comparison of the 2 models suggested attenuation of the individual-level estimates in the group models (refer to the part of Figure 4 highlighted with an asterisk). Even with random effects by participant, in the MLM, these estimates are, by design, biased by the group and may underestimate individual-level effects, especially for uncommon phenotypes [ 20 ]. As will be revealed later in the results from the larger sample of target participants, group-based estimation can even result in the reversal of the sign of the association for some individuals.

The iLM Is a Valid Framework to Identify Individual-Level Associations

Proof-of-concept analyses of the pilot data demonstrated that the iLM successfully captures expected associations between objective behavior and psychological variables, has an adequate model specification, and captures individual-level estimates unbiased by the group. Moreover, point estimates obtained with our Bayesian iLMs are nearly identical to those obtained through iLMs fitted with a frequentist inference framework (refer to Figure S2 in Multimedia Appendix 1 ), suggesting that our individual-level modeling approach is robust across both major statistical inference frameworks. In sum, the iLM is a valid framework to identify individual-level associations, which justified carrying over the model to test target hypotheses regarding the association between perceived stress and objective sleep duration in the independent target data set of first-year students (n=49).

Sleep and Stress Fluctuate With the Academic Calendar in the Target Participants

As with the pilot participants, during winter break and weekends, participants had longer objective sleep duration, better subjective sleep quality, and felt less stressed compared to the fall and spring semesters (columns 1 and 2 in Figure S3 in Multimedia Appendix 1 ). We replicated the autocorrelation pattern seen in the pilot data set: sleep duration showed the greatest autocorrelation at a 7-day lag (and then again at a 14-day lag), indicating that sleep patterns are tied to a weekly schedule, while stress showed the greatest autocorrelation at 1-day and 2-day lags, indicating that experiences of stress might come in chains of a few days (column 3 in Figure S3 in Multimedia Appendix 1 ). These school break– and week-related dynamics confirm that our target data set captured real-life dynamics associated with college life and reinforce our individual-level modeling decisions regarding the exclusion of winter break data and the addition of the day of the week as a covariate.

Shorter Objective Sleep Duration Is Associated With Worse Subjective Sleep Quality in the Target Participants

All participants showed a positive slope estimate for the association between concurrent sleep duration and sleep quality, and this effect was statistically significant in 46 (94%) of the 49 target participants (pd>0.975; Table 3 and Figure 5 A, in orange). In other words, when participants slept for a shorter period than usual, they also rated these same sleep events as worse quality, demonstrating that the iLM can capture expected real-world relations between behavioral and psychological phenomena. Among participants who showed a significant association (46/49, 94%), a 1-point increase in a 5-point sleep quality scale was associated with a sleep episode that was also longer by a median 59 (range 16-119) minutes, a substantial increase in sleep duration for most participants considering that the average sleep duration in the sample was 432 minutes (7.2 h; SD 34 min).

For the exploratory lagged model, the expected negative association between sleep duration and sleep quality the day before reached statistical significance in only 5 (10%) of the 49 participants (pd>0.975; Table 4 and Figure 5 A, in blue). In other words, for only 10% of participants, sleep rated as worse quality was consistently followed by longer sleep the following night, suggesting that sleep rebound effects are perhaps less common or reliable than anticipated. Among these participants, a 1-point decrease in a 5-point sleep quality scale was associated with a subsequent sleep episode that was longer by a median 18 (range 15-22) minutes across participants, a much smaller effect compared to the concurrent model. We did not observe structured patterns of association between individual slope estimates and individuals’ mean sleep quality, mean sleep duration, or total number of daily observations, suggesting that the model results were not systematically influenced by person-level aggregates of the variables that went into the model ( Figure 5 B).

Higher Subjective Stress Is Associated With Shorter Objective Sleep Duration in Most Target Participants, but the Direction of the Temporal Association Varies

The slope estimate for the association between stress and sleep duration was negative in 86% of all iLMs fitted such that an increase in stress was associated with a decrease in sleep duration ( Tables 5 and 6 ; Figure 6 A). This association between stress and sleep duration reached statistical significance in 19 (39%) of the 49 participants (pd>0.975); of these 19 participants, 18 (95%) showed a negative association. Of note, the only participant who showed a significant positive association (T15, for the association between sleep duration and stress the day after) also had no usable actigraphy data over the full spring semester due to a technical issue with their wristband, and their estimated positive slope should therefore be interpreted with caution considering the structured missingness in their data. No other participant in the target sample showed this kind of systematic missingness in either the actigraphy or survey data streams (refer to Figure S1 in Multimedia Appendix 1 ).

Of the 19 participants who showed a statistically significant relationship between stress and sleep duration, 8 (42%) showed only the stress-then-sleep phenotype, that is, days with shorter sleep durations were preceded by higher stress the day before ( Figure 6 , in green) but not vice versa; 5 (26%) showed only the sleep-then-stress phenotype, that is, nights with shorter sleep duration were followed by increased stress the day after ( Figure 6 , in purple); and 6 (32%) showed bidirectional effects such that nights with shorter sleep duration were preceded by increased stress the day before as well as followed by increased stress the day after. Among participants who showed a significant association between today’s stress levels and sleep duration later that night (14/49, 29%), a 1-point increase in a 5-point perceived stress scale was associated with shorter subsequent sleep duration of a median 17 (range 11-33) minutes across participants. Among participants who showed a significant association between today’s stress levels and last night’s sleep duration (11/49, 22%), a 1-point increase in a 5-point perceived stress scale was associated with shorter previous sleep duration of a median 18 (range 10-38) minutes across participants. These effects would compound to more substantial reductions in sleep duration with greater increases in daily stress.

Individual slope estimates of the association between sleep duration and stress showed no clear pattern of association with individuals’ mean sleep duration or the number of daily observations that went into the model (see Figure 6 B for mean sleep duration and number of daily observations). Individuals with higher mean stress tended to have a larger absolute slope estimate (see Figure 6 B for mean stress), perhaps because participants with very low stress levels have little variance to be modeled.

Person-Specific Estimates Get Attenuated in Group-Based Modeling

A comparison of the estimates obtained through the iLM and those of an MLM demonstrates that individual-level estimates get systematically attenuated in a group-based approach when there is between-person heterogeneity in the effects ( Figure 7 ). When there was a strong effect and small between-person variability in the tested associations, as with the association between sleep duration and concurrent sleep quality, the iLMs provided slope estimates nearly identical to those estimated through an MLM ( Figure 7 A). However, for associations that showed a weaker effect and greater degree of between-person variability, group-level approaches systematically flatten individually tailored effect sizes or even reverse the sign of the association ( Figure 7 B). We observed this in the lead-lag associations between stress and sleep duration (as highlighted with asterisks in Figure 7 ) and to some degree in the lagged association between sleep quality and sleep duration.

Examining Raw Within-Individual Data Informs the iLM Results

A closer look at individual participants’ data reveals important considerations for the application and interpretation of the iLMs ( Figure 8 ). As with other statistical models, the iLM is unlikely to detect stable associations for participants with too little variability in their data; for instance, participant T45 had consistently low stress levels over the course of the year, with occasional, small increases in stress tied to periods with increased academic demands (eg, ahead of midterm and final examinations periods). This participant’s individual linear models showed null results for both directions of stress-sleep associations (pd<0.975). Meanwhile, participant T29 also had low baseline stress levels, but they presented more frequent and substantial rises in stress throughout the year. Participant T29’s iLMs showed bidirectional negative associations between stress and sleep duration (pd>0.975).

While variability in daily observations over time is necessary to detect linear associations within the iLM, variability alone is not sufficient. Participant T10 showed substantial fluctuations in stress and sleep duration throughout the year, and their iLMs found bidirectional negative associations between stress and sleep duration (pds>0.975). Meanwhile, participant T16 also presented substantial variability in stress and sleep duration, and while their iLM detected a significant association between sleep duration and stress the day after (pd=1.00), no association was detected in the opposite direction (pd=0.65).

Principal Findings

Stress levels and sleep duration interact in an individual’s daily life, but research has yielded mixed findings regarding the temporal directionality of their associations, with changes in stress preceding changes in sleep, changes in sleep preceding changes in stress, or both. Here, we leveraged a novel individual-level linear regression modeling iLM framework to obtain precision estimates of day-to-day associations between self-reported stress levels and actigraphy-derived sleep duration in a sample of first-year college students studied continuously for a full academic year. While most of the participants (45/49, 92%) showed a negative association between daily stress levels and sleep duration, the temporal direction of the association varied, with all types of lead-lag association previously reported at the group level present within distinct individuals in our sample.

In agreement with prior literature, our results within individuals confirm that stress levels and sleep duration are closely and inversely related in daily life [ 10 - 19 ]. Nearly four-tenths (19/49, 39%) of the participants—each considered an independent test of the association—showed a statistically significant effect (in either temporal direction), revealing that day-to-day changes in stress levels or sleep duration can reliably predict one another in the real world in a substantial portion of the sample. The slope estimate for the association between stress and sleep duration was negative in 86% of the fitted models. Within the estimated slopes that reached statistical significance, all but one were negative. This suggests that, for most individuals, increased stress levels are associated with shorter rather than longer sleep duration in the surrounding days, consistent with findings showing that periods of heightened stress levels coincide with periods of reduced sleep [ 11 - 14 ]. Critically, we provide a framework to obtain individually tailored estimates of these day-to-day associations. The individual-level slope estimates ranged from 10 to 38 minutes in shorter sleep duration per 1-unit increase in a 5-point perceived stress scale, suggesting that a change in stress levels can be associated with substantial changes in sleep duration, especially when daily stress levels increased by several units. It should be noted that while negative associations between sleep duration and stress levels predominated in our sample, it is possible that some individuals in the wider population show positive associations but were not captured in our study.

Our precision approach further revealed that the temporal directionality of the association between stress and sleep duration varied from person to person, representing all patterns of results reported by prior, group-level studies. For some of the participants (8/49, 16%), heightened stress during the day associated with shorter sleep later that night but not vice versa, in agreement with group results reported in the studies by Marcusson-Clavertz et al [ 15 ] and Slavish et al [ 16 ]; for others (5/49, 10%), shorter sleep associated with heightened stress the next day but not vice versa, in agreement with group results in the study by Sin et al [ 17 ]; and yet others (6/49, 12%) showed both directions of association, in agreement with group results in the studies by Doane and Thurston [ 18 ] and Yap et al [ 19 ]. Daily psychological and behavioral experiences such as perceived stress and sleep duration thus seem to interact in person-specific ways, rather than being uniform across the population.

For individuals showing the stress–then–reduced-sleep phenotype, experiences of heightened stress (eg, due to an impending final examination or social conflict) might elicit hyperarousal and rumination [ 13 , 51 - 53 ], as well as behaviors aimed at mitigating the source of stress (eg, studying or socializing with friends late into the night), all of which can delay sleep and reduce its overall duration. Moreover, for those showing the reduced-sleep–then–stress phenotype, shortened sleep durations might enhance their sensitivity to (and undermine their ability to cope with) academic, interpersonal, or other stressors and thus make them more likely to experience heightened stress levels [ 10 , 54 , 55 ]. For some individuals, both patterns of effect might occur, with changes in stress levels and sleep duration reinforcing one another and resulting in chains of days with heightened stress and nights of short sleep that succeed one another.

Critically, our results suggest that group-level studies might report inconsistent findings, at least partly, because the dynamic interaction between daily stress levels and sleep duration varies from person to person. When data are aggregated at the group level, individual phenotypes might be obscured, and the resulting group-level estimates are suggestive of generalized effects when in fact they might only apply to a fraction of the sample. Even when hierarchical group models allow for fitting individual estimates, these are biased by the group and tend to attenuate (or shrink ) the estimation of individual effects [ 26 ]. A comparison of individual slope estimates of stress-sleep associations derived from our iLMs and those derived from an MLM with random intercepts and slopes starkly demonstrated this group bias: the group MLM estimated individual slopes that were systematically attenuated or even reversed sign compared to those estimated by our iLMs.

These results demonstrate the utility of individual-level modeling for characterizing real-world behavioral and psychological dynamics [ 27 - 29 , 34 ]. Our approach leverages mobile and wearable technology and provides a fit-for-purpose methodology that can turn these devices’ large-scale longitudinal measurements into meaningful insights. The iLM’s model specification is parsimonious by design and easy to interpret, including a single linear term for the main predictor of interest and a day-of-the-week covariate to account for the weekly structure in the outcome variable. Diagnostic checks confirmed that this simple model specification was powerful enough to capture real-world, stable linear associations between psychological phenomena and objective behaviors, while accounting for the time-related dependencies in the daily observations. Individual tailoring is achieved by fitting only 1 person’s data within a model, but the model specification remains identical across individuals, allowing for direct between-person comparisons of results, including estimating the relative prevalence of different phenotypes in the group.

The iLM framework is readily applied to a single individual’s data, making it useful for multiple real-world purposes beyond fundamental research that are increasingly gaining interest in the fields of consumer health informatics and digital health [ 21 , 31 , 32 ]. The iLM can be applied to data collected through personal devices for self-monitoring as well as for precision approaches in health care settings, where clinicians might use a patient’s data to triage intervention plans. Taking these results as an example, stress management interventions might be first prioritized among individuals for whom heightened stress precedes shortened sleep, while sleep interventions might be prioritized among individuals for whom shortened sleep precedes heightened stress. Moreover, this work might inform a growing body of research and products combining multiple data streams from wearables and smartphones along with machine learning techniques to predict experiences of stress [ 56 , 57 ]. Although prediction was not the focus of our work, understanding the person-specific association between stress and sleep duration, as well as the weekly behavioral patterns revealed by our actigraphy and survey data and individualized approach, could potentially contribute to the identification of periods when individuals are more likely to experience stress so that timely interventions can be offered.

The iLM offers a simple yet powerful precision framework for the estimation of real-world psychological and behavioral associations within the individual, but the results should be interpreted carefully in light of its assumptions and limitations. First, the iLM’s assumptions of linear, stable associations between the predictor and outcome variables are a deliberate attempt to simplify real-world behavioral and psychological dynamics that are highly complex. In the context of this study, these features allowed us to estimate college students’ day-to-day stress-sleep associations that are stable across the fluctuating demands on students within the school semesters as well as straightforward to interpret. However, it is possible that the association between stress and sleep duration is context dependent, varying as a function of the specific source of stress experienced by the individual (eg, academic vs interpersonal) and the broader seasonal demands (eg, whether school is in session). In fact, given the possibility of the latter, we decided a priori to exclude data collected during the winter break from our models. Future research could examine how these contextual demands influence stress-sleep associations, as well as explore nonlinear associations or cumulative effects over time.

Our analyses leveraged leading and lagging patterns in each individual’s time series of stress and sleep duration measures to ascertain the temporal directionality of their association (eg, stress during the day as a predictor of subsequent sleep duration later that night), but we did not implement a controlled experimental manipulation and cannot establish a causal relationship. While our use of intensive longitudinal data collected in the real world grants our results ecological validity, it also exposes them to confounders; for example, it is possible that a significant association between short sleep and higher stress the next day could be explained by the anticipated demands of the next day, such as an examination. Rather than short sleep duration causing higher stress the next day, the examination might be the primary cause behind both the short sleep (staying up late to prepare) and the stress reported the next day (heightened stress during test taking).

While our current intensive longitudinal data set and individual-level modeling framework passed sanity checks and diagnostics that confirmed data quality and adequate model specification, researchers applying our framework to other data sets and research questions should scrutinize the appropriateness of the data and the model before interpreting the results. Mobile and wearable technologies enable the collection of large behavioral data sets over time, but quantity does not guarantee quality, and long study periods require extra vigilance to ensure that participants remain compliant over time. Quality checks should confirm that the data collected are capturing expected real-world behavior (eg, as suggested by the intuitive changes we observed in students’ behavior between school terms and breaks as well as between weekdays and weekends). Even if the available data are substantial, and participant compliance is high, small variability in the metrics under study could still impede obtaining meaningful estimates of their associations, as demonstrated by participants with minimal fluctuations in stress levels over the course of the academic year. Moreover, investigators should be careful to identify appropriate uses, sanity checks, and interpretations of the iLM for their population of study; for example, part of our iLM validation process included testing the expected positive association between objective sleep duration (measured via the actigraphy wristband) and the participant’s subjective rating for the same sleep event (reported via a daily smartphone survey). While we expect that generally healthy participants will tend to rate nights of shorter-than-usual sleep duration as lower quality, it might not always be advisable to assume a simple linear association between objective sleep duration and subjective sleep quality, especially when modeling data from patients with sleep and psychiatric disorders.

Conclusions

Our novel iLM framework leveraged intensive longitudinal data from mobile and wearable devices to obtain individually tailored estimates of day-to-day associations between subjective stress levels and objective sleep duration. While stress and sleep duration were inversely related in most of the participants (45/49, 92%), the iLM revealed that the temporal direction of these associations is person specific, identifying a variety of individual phenotypes that may account for the diverse group-level findings reported in prior literature. Our results demonstrate the utility of individual-level modeling approaches for the assessment of behavioral and psychological associations in the real world. An individualized approach offers a foundation for the characterization of life dynamics at both the individual and group levels, as well as for the development of precision health and well-being interventions tailored to the individual.

Acknowledgments

The authors thank Laura Farfel, Marisa Marotta, Erin Phlegar, Lauren DiNicola, and Arpi Youssoufian for their help collecting data. Timothy O’Keefe, Harris Hoke, and Lily Jeong provided valuable assistance in neuroinformatics support. Kenzie W Carlson provided support with Beiwe. Katherine Miclau, Amira Song, Emily Iannazzi, and Hannah Becker helped with the actigraphy quality control procedure. This work was supported by a generous gift from Kent and Liz Dauten, National Institute of Mental Health grants (U01MH116925 and DP2MH103909), a grant from the National Institutes of Health (T90DA022759), and the Sackler Scholar Program in Psychobiology.

Data Availability

The data sets generated during this study are not publicly available due to concerns related to participant identifiability but may be available from the corresponding author on reasonable request.

Authors' Contributions

GCIII, J-PO, JTB, and RLB designed the research. GCIII, J-PO, and RLB performed the research. CMVB, GCIII, HR-E, and RLB analyzed the data. PM provided statistical support. CMVB and RLB wrote the paper, and all other authors reviewed the final manuscript.

Conflicts of Interest

J-PO is a cofounder and board member of Phebe Health, a commercial entity that operates in digital phenotyping. JTB has received consulting fees from Verily Life Sciences as well as consulting fees and equity from Mindstrong for work unrelated to this study. RLB has received consulting fees from Pfizer, Roche, Alkermes, and Cognito for work unrelated to this study. All other authors declare no other conflicts of interest.

Additional figures show participant-level missing data, comparison between Bayesian and frequentist individual-level linear model estimates, and group-level sleep and stress metrics in the target sample.

• Dyson R, Renk K. Freshmen adaptation to university life: depressive symptoms, stress, and coping. J Clin Psychol. Oct 2006;62(10):1231-1244. [ CrossRef ] [ Medline ]
• Ross SE, Niebling C, Heckert TM. Sources of stress among college students. Coll Stud J. 1999;33(2):312-318.
• Taylor ZE, Doane LD, Eisenberg N. Transitioning from high school to college: relations of social support, ego-resiliency, and maladjustment during emerging adulthood. Emerg Adulthood. Oct 21, 2013;2(2):105-115. [ CrossRef ]
• Bose M, Oliván B, Laferrère B. Stress and obesity: the role of the hypothalamic-pituitary-adrenal axis in metabolic disease. Curr Opin Endocrinol Diabetes Obes. Oct 2009;16(5):340-346. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kendler KS, Hettema JM, Butera F, Gardner CO, Prescott CA. Life event dimensions of loss, humiliation, entrapment, and danger in the prediction of onsets of major depression and generalized anxiety. Arch Gen Psychiatry. Aug 01, 2003;60(8):789-796. [ CrossRef ] [ Medline ]
• Luyster FS, Strollo PJJ, Zee PC, Walsh JK, Boards of Directors of the American Academy of Sleep Medicinethe Sleep Research Society. Sleep: a health imperative. Sleep. Jun 01, 2012;35(6):727-734. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pêgo JM, Sousa JC, Almeida OF, Sousa N. Stress and the neuroendocrinology of anxiety disorders. Curr Top Behav Neurosci. 2010;2:97-117. [ CrossRef ] [ Medline ]
• Rosengren A, Hawken S, Ounpuu S, Sliwa K, Zubaid M, Almahmeed WA, et al. INTERHEART investigators. Association of psychosocial risk factors with risk of acute myocardial infarction in 11119 cases and 13648 controls from 52 countries (the INTERHEART study): case-control study. Lancet. Sep 2004;364(9438):953-962. [ CrossRef ] [ Medline ]
• Zhang J, Paksarian D, Lamers F, Hickie IB, He J, Merikangas KR. Sleep patterns and mental health correlates in US adolescents. J Pediatr. Mar 2017;182:137-143. [ CrossRef ] [ Medline ]
• Goldstein AN, Walker MP. The role of sleep in emotional brain function. Annu Rev Clin Psychol. 2014;10:679-708. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kim EJ, Dimsdale JE. The effect of psychosocial stress on sleep: a review of polysomnographic evidence. Behav Sleep Med. Oct 29, 2007;5(4):256-278. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lund HG, Reider BD, Whiting AB, Prichard JR. Sleep patterns and predictors of disturbed sleep in a large population of college students. J Adolesc Health. Feb 2010;46(2):124-132. [ CrossRef ] [ Medline ]
• Sadeh A, Gruber R. Stress and sleep in adolescence: a clinical-developmental perspective. In: Carskadon MA, editor. Adolescent Sleep Patterns: Biological, Social, and Psychological Influences. Cambridge, UK. Cambridge University Press; 2002;236-253.
• Van Reeth O, Weibel L, Spiegel K, Leproult R, Dugovic C, Maccari S. Interactions between stress and sleep: from basic research to clinical situations. Sleep Med Rev. Apr 2000;4(2):201-219. [ FREE Full text ] [ CrossRef ]
• Marcusson-Clavertz D, Sliwinski MJ, Buxton OM, Kim J, Almeida DM, Smyth JM. Relationships between daily stress responses in everyday life and nightly sleep. J Behav Med. Aug 15, 2022;45(4):518-532. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Slavish DC, Asbee J, Veeramachaneni K, Messman BA, Scott B, Sin NL, et al. The cycle of daily stress and sleep: sleep measurement matters. Ann Behav Med. May 06, 2021;55(5):413-423. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sin NL, Almeida DM, Crain TL, Kossek EE, Berkman LF, Buxton OM. Bidirectional, temporal associations of sleep with positive events, affect, and stressors in daily life across a week. Ann Behav Med. Jun 10, 2017;51(3):402-415. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Doane LD, Thurston EC. Associations among sleep, daily experiences, and loneliness in adolescence: evidence of moderating and bidirectional pathways. J Adolesc. Feb 25, 2014;37(2):145-154. [ CrossRef ] [ Medline ]
• Yap Y, Slavish DC, Taylor DJ, Bei B, Wiley JF. Bi-directional relations between stress and self-reported and actigraphy-assessed sleep: a daily intensive longitudinal study. Sleep. Mar 12, 2020;43(3):a. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Harari GM, Lane ND, Wang R, Crosier BS, Campbell AT, Gosling SD. Using smartphones to collect behavioral data in psychological science: opportunities, practical considerations, and challenges. Perspect Psychol Sci. Nov 2016;11(6):838-854. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Onnela JP, Rauch SL. Harnessing smartphone-based digital phenotyping to enhance behavioral and mental health. Neuropsychopharmacology. Jun 2016;41(7):1691-1696. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Torous J, Kiang MV, Lorme J, Onnela JP. New tools for new research in psychiatry: a scalable and customizable platform to empower data driven smartphone research. JMIR Ment Health. May 05, 2016;3(2):e16. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sano A, Taylor S, McHill AW, Phillips AJ, Barger LK, Klerman E, et al. Identifying objective physiological markers and modifiable behaviors for self-reported stress and mental health status using wearable sensors and mobile phones: observational study. J Med Internet Res. Jun 08, 2018;20(6):e210. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vidal Bustamante CM, Coombs 3rd G, Rahimi-Eichi H, Mair P, Onnela JP, Baker JT, et al. Fluctuations in behavior and affect in college students measured using deep phenotyping. Sci Rep. Feb 04, 2022;12(1):1932. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wang R, Chen F, Chen Z, Li T, Harari G, Tignor S, et al. StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 2014. Presented at: UbiComp '14; September 13-17, 2014; Seattle, WA. [ CrossRef ]
• Raudenbush SW, Bryk AS. Hierarchical Linear Models: Applications and Data Analysis Methods. Thousand Oaks, CA. SAGE Publications, Inc; 2002.
• Barlow DH, Nock M, Hersen M. Single Case Experimental Designs: Strategies for Studying Behavior for Change. Boston, MA. Pearson/Allyn and Bacon; 2009.
• Haynes SN, Mumma GH, Pinson C. Idiographic assessment: conceptual and psychometric foundations of individualized behavioral assessment. Clin Psychol Rev. Mar 2009;29(2):179-191. [ CrossRef ] [ Medline ]
• Molenaar PC. A manifesto on psychology as idiographic science: bringing the person back into scientific psychology, this time forever. Meas Interdiscip Res Perspect. Oct 2004;2(4):201-218. [ CrossRef ]
• Piccirillo ML, Rodebaugh TL. Foundations of idiographic methods in psychology and applications for psychotherapy. Clin Psychol Rev. Jul 2019;71:90-100. [ CrossRef ] [ Medline ]
• Dunn J, Runge R, Snyder M. Wearables and the medical revolution. Per Med. Sep 2018;15(5):429-448. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fernandes BS, Williams LM, Steiner J, Leboyer M, Carvalho AF, Berk M. The new field of 'precision psychiatry'. BMC Med. Apr 13, 2017;15(1):80. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Scala JJ, Ganz AB, Snyder MP. Precision medicine approaches to mental health care. Physiology (Bethesda). Mar 01, 2023;38(2). [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wright AG, Woods WC. Personalized models of psychopathology. Annu Rev Clin Psychol. May 07, 2020;16:49-74. [ CrossRef ] [ Medline ]
• Onnela JP, Dixon C, Griffin K, Jaenicke T, Minowada L, Esterkin S, et al. Beiwe: a data collection platform for high-throughput digital phenotyping. J Open Source Softw. Dec 2021;6(68):3417. [ CrossRef ]
• Rahimi-Eichi H, Coombs III G, Vidal Bustamante CM, Onnela JP, Baker JT, Buckner RL. Open-source longitudinal sleep analysis from accelerometer data (DPSleep): algorithm development and validation. JMIR Mhealth Uhealth. Oct 06, 2021;9(10):e29849. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pickup M. Introduction to Time Series Analysis. Thousand Oaks, CA. SAGE Publications, Inc; 2015.
• Muth C, Oravecz Z, Gabry J. User-friendly Bayesian regression modeling: a tutorial with rstanarm and shinystan. Quant Method Psychol. 2018;14(2):99-119. [ CrossRef ]
• McElreath R. Statistical Rethinking: A Bayesian Course with Examples in R and Stan. Boca Raton, FL. CRC Press; 2018.
• Kruschke JK, Liddell TM. The Bayesian New Statistics: hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective. Psychon Bull Rev. Feb 7, 2018;25(1):178-206. [ CrossRef ] [ Medline ]
• R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing. 2019. URL: https://www.R-project.org/ [accessed 2024-04-04]
• Stan Development Team. The Stan Core Library. 2018. URL: http://mc-stan.org [accessed 2024-04-04]
• Goodrich B, Gabry J, Ali I, Brilleman S. rstanarm: Bayesian applied regression modeling via Stan. R package version 2.21.4. URL: https://mc-stan.org/rstanarm/ [accessed 2024-03-17]
• Kay M. tidybayes: tidy data and Geoms for Bayesian models. GitHub. 2023. URL: http://mjskay.github.io/tidybayes/ [accessed 2024-04-04]
• Makowski D, Ben-Shachar MS, Lüdecke D. bayestestR: describing effects and their uncertainty, existence and significance within the Bayesian framewor. J Open Source Softw. Aug 2019;4(40):1541. [ CrossRef ]
• Gelman A, Rubin DB. Inference from iterative simulation using multiple sequences. Statist Sci. Nov 1, 1992;7(4):457-472. [ CrossRef ]
• Gelman A, Carlin JB, Stern HS, Rubin DB, Vehtari A, Rubin DB. Bayesian Data Analysis, Third Edition. Boca Raton, FL. CRC Press; 2013.
• Makowski D, Ben-Shachar MS, Chen SH, Lüdecke D. Indices of effect existence and significance in the Bayesian framework. Front Psychol. 2019;10:2767. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kruschke JK. Rejecting or accepting parameter values in Bayesian estimation. Adv Method Pract Psychol Sci. May 08, 2018;1(2):270-280. [ CrossRef ]
• Cohen J. Statistical Power Analysis for the Behavioral Sciences Second Edition. New York, NY. Routledge; 1988.
• Drake CL, Pillai V, Roth T. Stress and sleep reactivity: a prospective investigation of the stress-diathesis model of insomnia. Sleep. Aug 01, 2014;37(8):1295-1304. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Harvey AG, Tang NK, Browning L. Cognitive approaches to insomnia. Clin Psychol Rev. Jul 2005;25(5):593-611. [ CrossRef ] [ Medline ]
• Kalmbach DA, Cuamatzi-Castelan AS, Tonnu CV, Tran KM, Anderson JR, Roth T, et al. Hyperarousal and sleep reactivity in insomnia: current insights. Nature Sci Sleep. Jul 2018;10:193-201. [ CrossRef ]
• Minkel JD, Banks S, Htaik O, Moreta MC, Jones CW, McGlinchey EL, et al. Sleep deprivation and stressors: evidence for elevated negative affect in response to mild stressors when sleep deprived. Emotion. Oct 2012;12(5):1015-1020. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zohar D, Tzischinsky O, Epstein R, Lavie P. The effects of sleep loss on medical residents' emotional reactions to work events: a cognitive-energy model. Sleep. Jan 2005;28(1):47-54. [ CrossRef ] [ Medline ]
• Vos G, Trinh K, Sarnyai Z, Rahimi Azghadi M. Generalizable machine learning for stress monitoring from wearable devices: a systematic literature review. Int J Med Inform. May 2023;173:105026. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Razavi M, Ziyadidegan S, Jahromi R, Kazeminasab S, Baharlouei E, Janfaza V, et al. Machine learning, deep learning and data preprocessing techniques for detection, prediction, and monitoring of stress and stress-related mental disorders: a scoping review. JMIR Preprints. [ FREE Full text ] [ CrossRef ]

Abbreviations

Edited by A Mavragani; submitted 26.10.23; peer-reviewed by R Meng, G Vos; comments to author 05.12.23; revised version received 10.01.24; accepted 07.03.24; published 30.04.24.

©Constanza M Vidal Bustamante, Garth Coombs III, Habiballah Rahimi-Eichi, Patrick Mair, Jukka-Pekka Onnela, Justin T Baker, Randy L Buckner. Originally published in JMIR Formative Research (https://formative.jmir.org), 30.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.