a research paper on education

• Research article
• Open access
• Published: 06 February 2017

Blended learning effectiveness: the relationship between student characteristics, design features and outcomes

• Mugenyi Justice Kintu   ORCID: orcid.org/0000-0002-4500-1168 1 , 2 ,
• Chang Zhu 2 &
• Edmond Kagambe 1  

International Journal of Educational Technology in Higher Education volume  14 , Article number:  7 ( 2017 ) Cite this article

768k Accesses

228 Citations

37 Altmetric

Metrics details

This paper investigates the effectiveness of a blended learning environment through analyzing the relationship between student characteristics/background, design features and learning outcomes. It is aimed at determining the significant predictors of blended learning effectiveness taking student characteristics/background and design features as independent variables and learning outcomes as dependent variables. A survey was administered to 238 respondents to gather data on student characteristics/background, design features and learning outcomes. The final semester evaluation results were used as a measure for performance as an outcome. We applied the online self regulatory learning questionnaire for data on learner self regulation, the intrinsic motivation inventory for data on intrinsic motivation and other self-developed instruments for measuring the other constructs. Multiple regression analysis results showed that blended learning design features (technology quality, online tools and face-to-face support) and student characteristics (attitudes and self-regulation) predicted student satisfaction as an outcome. The results indicate that some of the student characteristics/backgrounds and design features are significant predictors for student learning outcomes in blended learning.

Introduction

The teaching and learning environment is embracing a number of innovations and some of these involve the use of technology through blended learning. This innovative pedagogical approach has been embraced rapidly though it goes through a process. The introduction of blended learning (combination of face-to-face and online teaching and learning) initiatives is part of these innovations but its uptake, especially in the developing world faces challenges for it to be an effective innovation in teaching and learning. Blended learning effectiveness has quite a number of underlying factors that pose challenges. One big challenge is about how users can successfully use the technology and ensuring participants’ commitment given the individual learner characteristics and encounters with technology (Hofmann, 2014 ). Hofmann adds that users getting into difficulties with technology may result into abandoning the learning and eventual failure of technological applications. In a report by Oxford Group ( 2013 ), some learners (16%) had negative attitudes to blended learning while 26% were concerned that learners would not complete study in blended learning. Learners are important partners in any learning process and therefore, their backgrounds and characteristics affect their ability to effectively carry on with learning and being in blended learning, the design tools to be used may impinge on the effectiveness in their learning.

This study tackles blended learning effectiveness which has been investigated in previous studies considering grades, course completion, retention and graduation rates but no studies regarding effectiveness in view of learner characteristics/background, design features and outcomes have been done in the Ugandan university context. No studies have also been done on how the characteristics of learners and design features are predictors of outcomes in the context of a planning evaluation research (Guskey, 2000 ) to establish the effectiveness of blended learning. Guskey ( 2000 ) noted that planning evaluation fits in well since it occurs before the implementation of any innovation as well as allowing planners to determine the needs, considering participant characteristics, analyzing contextual matters and gathering baseline information. This study is done in the context of a plan to undertake innovative pedagogy involving use of a learning management system (moodle) for the first time in teaching and learning in a Ugandan university. The learner characteristics/backgrounds being investigated for blended learning effectiveness include self-regulation, computer competence, workload management, social and family support, attitude to blended learning, gender and age. We investigate the blended learning design features of learner interactions, face-to-face support, learning management system tools and technology quality while the outcomes considered include satisfaction, performance, intrinsic motivation and knowledge construction. Establishing the significant predictors of outcomes in blended learning will help to inform planners of such learning environments in order to put in place necessary groundwork preparations for designing blended learning as an innovative pedagogical approach.

Kenney and Newcombe ( 2011 ) did their comparison to establish effectiveness in view of grades and found that blended learning had higher average score than the non-blended learning environment. Garrison and Kanuka ( 2004 ) examined the transformative potential of blended learning and reported an increase in course completion rates, improved retention and increased student satisfaction. Comparisons between blended learning environments have been done to establish the disparity between academic achievement, grade dispersions and gender performance differences and no significant differences were found between the groups (Demirkol & Kazu, 2014 ).

However, blended learning effectiveness may be dependent on many other factors and among them student characteristics, design features and learning outcomes. Research shows that the failure of learners to continue their online education in some cases has been due to family support or increased workload leading to learner dropout (Park & Choi, 2009 ) as well as little time for study. Additionally, it is dependent on learner interactions with instructors since failure to continue with online learning is attributed to this. In Greer, Hudson & Paugh’s study as cited in Park and Choi ( 2009 ), family and peer support for learners is important for success in online and face-to-face learning. Support is needed for learners from all areas in web-based courses and this may be from family, friends, co-workers as well as peers in class. Greer, Hudson and Paugh further noted that peer encouragement assisted new learners in computer use and applications. The authors also show that learners need time budgeting, appropriate technology tools and support from friends and family in web-based courses. Peer support is required by learners who have no or little knowledge of technology, especially computers, to help them overcome fears. Park and Choi, ( 2009 ) showed that organizational support significantly predicts learners’ stay and success in online courses because employers at times are willing to reduce learners’ workload during study as well as supervisors showing that they are interested in job-related learning for employees to advance and improve their skills.

The study by Kintu and Zhu ( 2016 ) investigated the possibility of blended learning in a Ugandan University and examined whether student characteristics (such as self-regulation, attitudes towards blended learning, computer competence) and student background (such as family support, social support and management of workload) were significant factors in learner outcomes (such as motivation, satisfaction, knowledge construction and performance). The characteristics and background factors were studied along with blended learning design features such as technology quality, learner interactions, and Moodle with its tools and resources. The findings from that study indicated that learner attitudes towards blended learning were significant factors to learner satisfaction and motivation while workload management was a significant factor to learner satisfaction and knowledge construction. Among the blended learning design features, only learner interaction was a significant factor to learner satisfaction and knowledge construction.

The focus of the present study is on examining the effectiveness of blended learning taking into consideration learner characteristics/background, blended learning design elements and learning outcomes and how the former are significant predictors of blended learning effectiveness.

Studies like that of Morris and Lim ( 2009 ) have investigated learner and instructional factors influencing learning outcomes in blended learning. They however do not deal with such variables in the contexts of blended learning design as an aspect of innovative pedagogy involving the use of technology in education. Apart from the learner variables such as gender, age, experience, study time as tackled before, this study considers social and background aspects of the learners such as family and social support, self-regulation, attitudes towards blended learning and management of workload to find out their relationship to blended learning effectiveness. Identifying the various types of learner variables with regard to their relationship to blended learning effectiveness is important in this study as we embark on innovative pedagogy with technology in teaching and learning.

Literature review

This review presents research about blended learning effectiveness from the perspective of learner characteristics/background, design features and learning outcomes. It also gives the factors that are considered to be significant for blended learning effectiveness. The selected elements are as a result of the researcher’s experiences at a Ugandan university where student learning faces challenges with regard to learner characteristics and blended learning features in adopting the use of technology in teaching and learning. We have made use of Loukis, Georgiou, and Pazalo ( 2007 ) value flow model for evaluating an e-learning and blended learning service specifically considering the effectiveness evaluation layer. This evaluates the extent of an e-learning system usage and the educational effectiveness. In addition, studies by Leidner, Jarvenpaa, Dillon and Gunawardena as cited in Selim ( 2007 ) have noted three main factors that affect e-learning and blended learning effectiveness as instructor characteristics, technology and student characteristics. Heinich, Molenda, Russell, and Smaldino ( 2001 ) showed the need for examining learner characteristics for effective instructional technology use and showed that user characteristics do impact on behavioral intention to use technology. Research has dealt with learner characteristics that contribute to learner performance outcomes. They have dealt with emotional intelligence, resilience, personality type and success in an online learning context (Berenson, Boyles, & Weaver, 2008 ). Dealing with the characteristics identified in this study will give another dimension, especially for blended learning in learning environment designs and add to specific debate on learning using technology. Lin and Vassar, ( 2009 ) indicated that learner success is dependent on ability to cope with technical difficulty as well as technical skills in computer operations and internet navigation. This justifies our approach in dealing with the design features of blended learning in this study.

Learner characteristics/background and blended learning effectiveness

Studies indicate that student characteristics such as gender play significant roles in academic achievement (Oxford Group, 2013 ), but no study examines performance of male and female as an important factor in blended learning effectiveness. It has again been noted that the success of e- and blended learning is highly dependent on experience in internet and computer applications (Picciano & Seaman, 2007 ). Rigorous discovery of such competences can finally lead to a confirmation of high possibilities of establishing blended learning. Research agrees that the success of e-learning and blended learning can largely depend on students as well as teachers gaining confidence and capability to participate in blended learning (Hadad, 2007 ). Shraim and Khlaif ( 2010 ) note in their research that 75% of students and 72% of teachers were lacking in skills to utilize ICT based learning components due to insufficient skills and experience in computer and internet applications and this may lead to failure in e-learning and blended learning. It is therefore pertinent that since the use of blended learning applies high usage of computers, computer competence is necessary (Abubakar & Adetimirin, 2015 ) to avoid failure in applying technology in education for learning effectiveness. Rovai, ( 2003 ) noted that learners’ computer literacy and time management are crucial in distance learning contexts and concluded that such factors are meaningful in online classes. This is supported by Selim ( 2007 ) that learners need to posses time management skills and computer skills necessary for effectiveness in e- learning and blended learning. Self-regulatory skills of time management lead to better performance and learners’ ability to structure the physical learning environment leads to efficiency in e-learning and blended learning environments. Learners need to seek helpful assistance from peers and teachers through chats, email and face-to-face meetings for effectiveness (Lynch & Dembo, 2004 ). Factors such as learners’ hours of employment and family responsibilities are known to impede learners’ process of learning, blended learning inclusive (Cohen, Stage, Hammack, & Marcus, 2012 ). It was also noted that a common factor in failure and learner drop-out is the time conflict which is compounded by issues of family , employment status as well as management support (Packham, Jones, Miller, & Thomas, 2004 ). A study by Thompson ( 2004 ) shows that work, family, insufficient time and study load made learners withdraw from online courses.

Learner attitudes to blended learning can result in its effectiveness and these shape behavioral intentions which usually lead to persistence in a learning environment, blended inclusive. Selim, ( 2007 ) noted that the learners’ attitude towards e-learning and blended learning are success factors for these learning environments. Learner performance by age and gender in e-learning and blended learning has been found to indicate no significant differences between male and female learners and different age groups (i.e. young, middle-aged and old above 45 years) (Coldwell, Craig, Paterson, & Mustard, 2008 ). This implies that the potential for blended learning to be effective exists and is unhampered by gender or age differences.

Blended learning design features

The design features under study here include interactions, technology with its quality, face-to-face support and learning management system tools and resources.

Research shows that absence of learner interaction causes failure and eventual drop-out in online courses (Willging & Johnson, 2009 ) and the lack of learner connectedness was noted as an internal factor leading to learner drop-out in online courses (Zielinski, 2000 ). It was also noted that learners may not continue in e- and blended learning if they are unable to make friends thereby being disconnected and developing feelings of isolation during their blended learning experiences (Willging & Johnson, 2009). Learners’ Interactions with teachers and peers can make blended learning effective as its absence makes learners withdraw (Astleitner, 2000 ). Loukis, Georgious and Pazalo (2007) noted that learners’ measuring of a system’s quality, reliability and ease of use leads to learning efficiency and can be so in blended learning. Learner success in blended learning may substantially be affected by system functionality (Pituch & Lee, 2006 ) and may lead to failure of such learning initiatives (Shrain, 2012 ). It is therefore important to examine technology quality for ensuring learning effectiveness in blended learning. Tselios, Daskalakis, and Papadopoulou ( 2011 ) investigated learner perceptions after a learning management system use and found out that the actual system use determines the usefulness among users. It is again noted that a system with poor response time cannot be taken to be useful for e-learning and blended learning especially in cases of limited bandwidth (Anderson, 2004 ). In this study, we investigate the use of Moodle and its tools as a function of potential effectiveness of blended learning.

The quality of learning management system content for learners can be a predictor of good performance in e-and blended learning environments and can lead to learner satisfaction. On the whole, poor quality technology yields no satisfaction by users and therefore the quality of technology significantly affects satisfaction (Piccoli, Ahmad, & Ives, 2001 ). Continued navigation through a learning management system increases use and is an indicator of success in blended learning (Delone & McLean, 2003 ). The efficient use of learning management system and its tools improves learning outcomes in e-learning and blended learning environments.

It is noted that learner satisfaction with a learning management system can be an antecedent factor for blended learning effectiveness. Goyal and Tambe ( 2015 ) noted that learners showed an appreciation to Moodle’s contribution in their learning. They showed positivity with it as it improved their understanding of course material (Ahmad & Al-Khanjari, 2011 ). The study by Goyal and Tambe ( 2015 ) used descriptive statistics to indicate improved learning by use of uploaded syllabus and session plans on Moodle. Improved learning is also noted through sharing study material, submitting assignments and using the calendar. Learners in the study found Moodle to be an effective educational tool.

In blended learning set ups, face-to-face experiences form part of the blend and learner positive attitudes to such sessions could mean blended learning effectiveness. A study by Marriot, Marriot, and Selwyn ( 2004 ) showed learners expressing their preference for face-to-face due to its facilitation of social interaction and communication skills acquired from classroom environment. Their preference for the online session was only in as far as it complemented the traditional face-to-face learning. Learners in a study by Osgerby ( 2013 ) had positive perceptions of blended learning but preferred face-to-face with its step-by-stem instruction. Beard, Harper and Riley ( 2004 ) shows that some learners are successful while in a personal interaction with teachers and peers thus prefer face-to-face in the blend. Beard however dealt with a comparison between online and on-campus learning while our study combines both, singling out the face-to-face part of the blend. The advantage found by Beard is all the same relevant here because learners in blended learning express attitude to both online and face-to-face for an effective blend. Researchers indicate that teacher presence in face-to-face sessions lessens psychological distance between them and the learners and leads to greater learning. This is because there are verbal aspects like giving praise, soliciting for viewpoints, humor, etc and non-verbal expressions like eye contact, facial expressions, gestures, etc which make teachers to be closer to learners psychologically (Kelley & Gorham, 2009 ).

Learner outcomes

The outcomes under scrutiny in this study include performance, motivation, satisfaction and knowledge construction. Motivation is seen here as an outcome because, much as cognitive factors such as course grades are used in measuring learning outcomes, affective factors like intrinsic motivation may also be used to indicate outcomes of learning (Kuo, Walker, Belland, & Schroder, 2013 ). Research shows that high motivation among online learners leads to persistence in their courses (Menager-Beeley, 2004 ). Sankaran and Bui ( 2001 ) indicated that less motivated learners performed poorly in knowledge tests while those with high learning motivation demonstrate high performance in academics (Green, Nelson, Martin, & Marsh, 2006 ). Lim and Kim, ( 2003 ) indicated that learner interest as a motivation factor promotes learner involvement in learning and this could lead to learning effectiveness in blended learning.

Learner satisfaction was noted as a strong factor for effectiveness of blended and online courses (Wilging & Johnson, 2009) and dissatisfaction may result from learners’ incompetence in the use of the learning management system as an effective learning tool since, as Islam ( 2014 ) puts it, users may be dissatisfied with an information system due to ease of use. A lack of prompt feedback for learners from course instructors was found to cause dissatisfaction in an online graduate course. In addition, dissatisfaction resulted from technical difficulties as well as ambiguous course instruction Hara and Kling ( 2001 ). These factors, once addressed, can lead to learner satisfaction in e-learning and blended learning and eventual effectiveness. A study by Blocker and Tucker ( 2001 ) also showed that learners had difficulties with technology and inadequate group participation by peers leading to dissatisfaction within these design features. Student-teacher interactions are known to bring satisfaction within online courses. Study results by Swan ( 2001 ) indicated that student-teacher interaction strongly related with student satisfaction and high learner-learner interaction resulted in higher levels of course satisfaction. Descriptive results by Naaj, Nachouki, and Ankit ( 2012 ) showed that learners were satisfied with technology which was a video-conferencing component of blended learning with a mean of 3.7. The same study indicated student satisfaction with instructors at a mean of 3.8. Askar and Altun, ( 2008 ) found that learners were satisfied with face-to-face sessions of the blend with t-tests and ANOVA results indicating female scores as higher than for males in the satisfaction with face-to-face environment of the blended learning.

Studies comparing blended learning with traditional face-to-face have indicated that learners perform equally well in blended learning and their performance is unaffected by the delivery method (Kwak, Menezes, & Sherwood, 2013 ). In another study, learning experience and performance are known to improve when traditional course delivery is integrated with online learning (Stacey & Gerbic, 2007 ). Such improvement as noted may be an indicator of blended learning effectiveness. Our study however, delves into improved performance but seeks to establish the potential of blended learning effectiveness by considering grades obtained in a blended learning experiment. Score 50 and above is considered a pass in this study’s setting and learners scoring this and above will be considered to have passed. This will make our conclusions about the potential of blended learning effectiveness.

Regarding knowledge construction, it has been noted that effective learning occurs where learners are actively involved (Nurmela, Palonen, Lehtinen & Hakkarainen, 2003 , cited in Zhu, 2012 ) and this may be an indicator of learning environment effectiveness. Effective blended learning would require that learners are able to initiate, discover and accomplish the processes of knowledge construction as antecedents of blended learning effectiveness. A study by Rahman, Yasin and Jusoff ( 2011 ) indicated that learners were able to use some steps to construct meaning through an online discussion process through assignments given. In the process of giving and receiving among themselves, the authors noted that learners learned by writing what they understood. From our perspective, this can be considered to be accomplishment in the knowledge construction process. Their study further shows that learners construct meaning individually from assignments and this stage is referred to as pre-construction which for our study, is an aspect of discovery in the knowledge construction process.

Predictors of blended learning effectiveness

Researchers have dealt with success factors for online learning or those for traditional face-to-face learning but little is known about factors that predict blended learning effectiveness in view of learner characteristics and blended learning design features. This part of our study seeks to establish the learner characteristics/backgrounds and design features that predict blended learning effectiveness with regard to satisfaction, outcomes, motivation and knowledge construction. Song, Singleton, Hill, and Koh ( 2004 ) examined online learning effectiveness factors and found out that time management (a self-regulatory factor) was crucial for successful online learning. Eom, Wen, and Ashill ( 2006 ) using a survey found out that interaction, among other factors, was significant for learner satisfaction. Technical problems with regard to instructional design were a challenge to online learners thus not indicating effectiveness (Song et al., 2004 ), though the authors also indicated that descriptive statistics to a tune of 75% and time management (62%) impact on success of online learning. Arbaugh ( 2000 ) and Swan ( 2001 ) indicated that high levels of learner-instructor interaction are associated with high levels of user satisfaction and learning outcomes. A study by Naaj et al. ( 2012 ) indicated that technology and learner interactions, among other factors, influenced learner satisfaction in blended learning.

Objective and research questions of the current study

The objective of the current study is to investigate the effectiveness of blended learning in view of student satisfaction, knowledge construction, performance and intrinsic motivation and how they are related to student characteristics and blended learning design features in a blended learning environment.

Research questions

What are the student characteristics and blended learning design features for an effective blended learning environment?

Which factors (among the learner characteristics and blended learning design features) predict student satisfaction, learning outcomes, intrinsic motivation and knowledge construction?

Conceptual model of the present study

The reviewed literature clearly shows learner characteristics/background and blended learning design features play a part in blended learning effectiveness and some of them are significant predictors of effectiveness. The conceptual model for our study is depicted as follows (Fig.  1 ):

Conceptual model of the current study

Research design

This research applies a quantitative design where descriptive statistics are used for the student characteristics and design features data, t-tests for the age and gender variables to determine if they are significant in blended learning effectiveness and regression for predictors of blended learning effectiveness.

This study is based on an experiment in which learners participated during their study using face-to-face sessions and an on-line session of a blended learning design. A learning management system (Moodle) was used and learner characteristics/background and blended learning design features were measured in relation to learning effectiveness. It is therefore a planning evaluation research design as noted by Guskey ( 2000 ) since the outcomes are aimed at blended learning implementation at MMU. The plan under which the various variables were tested involved face-to-face study at the beginning of a 17 week semester which was followed by online teaching and learning in the second half of the semester. The last part of the semester was for another face-to-face to review work done during the online sessions and final semester examinations. A questionnaire with items on student characteristics, design features and learning outcomes was distributed among students from three schools and one directorate of postgraduate studies.

Participants

Cluster sampling was used to select a total of 238 learners to participate in this study. Out of the whole university population of students, three schools and one directorate were used. From these, one course unit was selected from each school and all the learners following the course unit were surveyed. In the school of Education ( n  = 70) and Business and Management Studies ( n  = 133), sophomore students were involved due to the fact that they have been introduced to ICT basics during their first year of study. Students of the third year were used from the department of technology in the School of Applied Sciences and Technology ( n  = 18) since most of the year two courses had a lot of practical aspects that could not be used for the online learning part. From the Postgraduate Directorate ( n  = 17), first and second year students were selected because learners attend a face-to-face session before they are given paper modules to study away from campus.

The study population comprised of 139 male students representing 58.4% and 99 females representing 41.6% with an average age of 24 years.

Instruments

The end of semester results were used to measure learner performance. The online self-regulated learning questionnaire (Barnard, Lan, To, Paton, & Lai, 2009 ) and the intrinsic motivation inventory (Deci & Ryan, 1982 ) were applied to measure the constructs on self regulation in the student characteristics and motivation in the learning outcome constructs. Other self-developed instruments were used for the other remaining variables of attitudes, computer competence, workload management, social and family support, satisfaction, knowledge construction, technology quality, interactions, learning management system tools and resources and face-to-face support.

Instrument reliability

Cronbach’s alpha was used to test reliability and the table below gives the results. All the scales and sub-scales had acceptable internal consistency reliabilities as shown in Table  1 below:

Data analysis

First, descriptive statistics was conducted. Shapiro-Wilk test was done to test normality of the data for it to qualify for parametric tests. The test results for normality of our data before the t- test resulted into significant levels (Male = .003, female = .000) thereby violating the normality assumption. We therefore used the skewness and curtosis results which were between −1.0 and +1.0 and assumed distribution to be sufficiently normal to qualify the data for a parametric test, (Pallant, 2010 ). An independent samples t -test was done to find out the differences in male and female performance to explain the gender characteristics in blended learning effectiveness. A one-way ANOVA between subjects was conducted to establish the differences in performance between age groups. Finally, multiple regression analysis was done between student variables and design elements with learning outcomes to determine the significant predictors for blended learning effectiveness.

Student characteristics, blended learning design features and learning outcomes ( RQ1 )

A t- test was carried out to establish the performance of male and female learners in the blended learning set up. This was aimed at finding out if male and female learners do perform equally well in blended learning given their different roles and responsibilities in society. It was found that male learners performed slightly better ( M  = 62.5) than their female counterparts ( M  = 61.1). An independent t -test revealed that the difference between the performances was not statistically significant ( t  = 1.569, df = 228, p  = 0.05, one tailed). The magnitude of the differences in the means is small with effect size ( d  = 0.18). A one way between subjects ANOVA was conducted on the performance of different age groups to establish the performance of learners of young and middle aged age groups (20–30, young & and 31–39, middle aged). This revealed a significant difference in performance (F(1,236 = 8.498, p < . 001).

Average percentages of the items making up the self regulated learning scale are used to report the findings about all the sub-scales in the learner characteristics/background scale. Results show that learner self-regulation was good enough at 72.3% in all the sub-scales of goal setting, environment structuring, task strategies, time management, help-seeking and self-evaluation among learners. The least in the scoring was task strategies at 67.7% and the highest was learner environment structuring at 76.3%. Learner attitude towards blended learning environment is at 76% in the sub-scales of learner autonomy, quality of instructional materials, course structure, course interface and interactions. The least scored here is attitude to course structure at 66% and their attitudes were high on learner autonomy and course interface both at 82%. Results on the learners’ computer competences are summarized in percentages in the table below (Table  2 ):

It can be seen that learners are skilled in word processing at 91%, email at 63.5%, spreadsheets at 68%, web browsers at 70.2% and html tools at 45.4%. They are therefore good enough in word processing and web browsing. Their computer confidence levels are reported at 75.3% and specifically feel very confident when it comes to working with a computer (85.7%). Levels of family and social support for learners during blended learning experiences are at 60.5 and 75% respectively. There is however a low score on learners being assisted by family members in situations of computer setbacks (33.2%) as 53.4% of the learners reported no assistance in this regard. A higher percentage (85.3%) is reported on learners getting support from family regarding provision of essentials for learning such as tuition. A big percentage of learners spend two hours on study while at home (35.3%) followed by one hour (28.2%) while only 9.7% spend more than three hours on study at home. Peers showed great care during the blended learning experience (81%) and their experiences were appreciated by the society (66%). Workload management by learners vis-à-vis studying is good at 60%. Learners reported that their workmates stand in for them at workplaces to enable them do their study in blended learning while 61% are encouraged by their bosses to go and improve their skills through further education and training. On the time spent on other activities not related to study, majority of the learners spend three hours (35%) while 19% spend 6 hours. Sixty percent of the learners have to answer to someone when they are not attending to other activities outside study compared to the 39.9% who do not and can therefore do study or those other activities.

The usability of the online system, tools and resources was below average as shown in the table below in percentages (Table  3 ):

However, learners became skilled at navigating around the learning management system (79%) and it was easy for them to locate course content, tools and resources needed such as course works, news, discussions and journal materials. They effectively used the communication tools (60%) and to work with peers by making posts (57%). They reported that online resources were well organized, user friendly and easy to access (71%) as well as well structured in a clear and understandable manner (72%). They therefore recommended the use of online resources for other course units in future (78%) because they were satisfied with them (64.3%). On the whole, the online resources were fine for the learners (67.2%) and useful as a learning resource (80%). The learners’ perceived usefulness/satisfaction with online system, tools, and resources was at 81% as the LMS tools helped them to communicate, work with peers and reflect on their learning (74%). They reported that using moodle helped them to learn new concepts, information and gaining skills (85.3%) as well as sharing what they knew or learned (76.4%). They enjoyed the course units (78%) and improved their skills with technology (89%).

Learner interactions were seen from three angles of cognitivism, collaborative learning and student-teacher interactions. Collaborative learning was average at 50% with low percentages in learners posting challenges to colleagues’ ideas online (34%) and posting ideas for colleagues to read online (37%). They however met oftentimes online (60%) and organized how they would work together in study during the face-to-face meetings (69%). The common form of communication medium frequently used by learners during the blended learning experience was by phone (34.5%) followed by whatsapp (21.8%), face book (21%), discussion board (11.8%) and email (10.9%). At the cognitive level, learners interacted with content at 72% by reading the posted content (81%), exchanging knowledge via the LMS (58.4%), participating in discussions on the forum (62%) and got course objectives and structure introduced during the face-to-face sessions (86%). Student-teacher interaction was reported at 71% through instructors individually working with them online (57.2%) and being well guided towards learning goals (81%). They did receive suggestions from instructors about resources to use in their learning (75.3%) and instructors provided learning input for them to come up with their own answers (71%).

The technology quality during the blended learning intervention was rated at 69% with availability of 72%, quality of the resources was at 68% with learners reporting that discussion boards gave right content necessary for study (71%) and the email exchanges containing relevant and much needed information (63.4%) as well as chats comprising of essential information to aid the learning (69%). Internet reliability was rated at 66% with a speed considered averagely good to facilitate online activities (63%). They however reported that there was intermittent breakdown during online study (67%) though they could complete their internet program during connection (63.4%). Learners eventually found it easy to download necessary materials for study in their blended learning experiences (71%).

Learner extent of use of the learning management system features was as shown in the table below in percentage (Table  4 ):

From the table, very rarely used features include the blog and wiki while very often used ones include the email, forum, chat and calendar.

The effectiveness of the LMS was rated at 79% by learners reporting that they found it useful (89%) and using it makes their learning activities much easier (75.2%). Moodle has helped learners to accomplish their learning tasks more quickly (74%) and that as a LMS, it is effective in teaching and learning (88%) with overall satisfaction levels at 68%. However, learners note challenges in the use of the LMS regarding its performance as having been problematic to them (57%) and only 8% of the learners reported navigation while 16% reported access as challenges.

Learner attitudes towards Face-to-face support were reported at 88% showing that the sessions were enjoyable experiences (89%) with high quality class discussions (86%) and therefore recommended that the sessions should continue in blended learning (89%). The frequency of the face-to-face sessions is shown in the table below as preferred by learners (Table  5 ).

Learners preferred face-to-face sessions after every month in the semester (33.6%) and at the beginning of the blended learning session only (27.7%).

Learners reported high intrinsic motivation levels with interest and enjoyment of tasks at 83.7%, perceived competence at 70.2%, effort/importance sub-scale at 80%, pressure/tension reported at 54%. The pressure percentage of 54% arises from learners feeling nervous (39.2%) and a lot of anxiety (53%) while 44% felt a lot of pressure during the blended learning experiences. Learners however reported the value/usefulness of blended learning at 91% with majority believing that studying online and face-to-face had value for them (93.3%) and were therefore willing to take part in blended learning (91.2%). They showed that it is beneficial for them (94%) and that it was an important way of studying (84.3%).

Learner satisfaction was reported at 81% especially with instructors (85%) high percentage reported on encouraging learner participation during the course of study 93%, course content (83%) with the highest being satisfaction with the good relationship between the objectives of the course units and the content (90%), technology (71%) with a high percentage on the fact that the platform was adequate for the online part of the learning (76%), interactions (75%) with participation in class at 79%, and face-to-face sessions (91%) with learner satisfaction high on face-to-face sessions being good enough for interaction and giving an overview of the courses when objectives were introduced at 92%.

Learners’ knowledge construction was reported at 78% with initiation and discovery scales scoring 84% with 88% specifically for discovering the learning points in the course units. The accomplishment scale in knowledge construction scored 71% and specifically the fact that learners were able to work together with group members to accomplish learning tasks throughout the study of the course units (79%). Learners developed reports from activities (67%), submitted solutions to discussion questions (68%) and did critique peer arguments (69%). Generally, learners performed well in blended learning in the final examination with an average pass of 62% and standard deviation of 7.5.

Significant predictors of blended learning effectiveness ( RQ 2)

A standard multiple regression analysis was done taking learner characteristics/background and design features as predictor variables and learning outcomes as criterion variables. The data was first tested to check if it met the linear regression test assumptions and results showed the correlations between the independent variables and each of the dependent variables (highest 0.62 and lowest 0.22) as not being too high, which indicated that multicollinearity was not a problem in our model. From the coefficients table, the VIF values ranged from 1.0 to 2.4, well below the cut off value of 10 and indicating no possibility of multicollinearity. The normal probability plot was seen to lie as a reasonably straight diagonal from bottom left to top right indicating normality of our data. Linearity was found suitable from the scatter plot of the standardized residuals and was rectangular in distribution. Outliers were no cause for concern in our data since we had only 1% of all cases falling outside 3.0 thus proving the data as a normally distributed sample. Our R -square values was at 0.525 meaning that the independent variables explained about 53% of the variance in overall satisfaction, motivation and knowledge construction of the learners. All the models explaining the three dependent variables of learner satisfaction, intrinsic motivation and knowledge construction were significant at the 0.000 probability level (Table  6 ).

From the table above, design features (technology quality and online tools and resources), and learner characteristics (attitudes to blended learning, self-regulation) were significant predictors of learner satisfaction in blended learning. This means that good technology with the features involved and the learner positive attitudes with capacity to do blended learning with self drive led to their satisfaction. The design features (technology quality, interactions) and learner characteristics (self regulation and social support), were found to be significant predictors of learner knowledge construction. This implies that learners’ capacity to go on their work by themselves supported by peers and high levels of interaction using the quality technology led them to construct their own ideas in blended learning. Design features (technology quality, online tools and resources as well as learner interactions) and learner characteristics (self regulation), significantly predicted the learners’ intrinsic motivation in blended learning suggesting that good technology, tools and high interaction levels with independence in learning led to learners being highly motivated. Finally, none of the independent variables considered under this study were predictors of learning outcomes (grade).

In this study we have investigated learning outcomes as dependent variables to establish if particular learner characteristics/backgrounds and design features are related to the outcomes for blended learning effectiveness and if they predict learning outcomes in blended learning. We took students from three schools out of five and one directorate of post-graduate studies at a Ugandan University. The study suggests that the characteristics and design features examined are good drivers towards an effective blended learning environment though a few of them predicted learning outcomes in blended learning.

Student characteristics/background, blended learning design features and learning outcomes

The learner characteristics, design features investigated are potentially important for an effective blended learning environment. Performance by gender shows a balance with no statistical differences between male and female. There are statistically significant differences ( p  < .005) in the performance between age groups with means of 62% for age group 20–30 and 67% for age group 31 –39. The indicators of self regulation exist as well as positive attitudes towards blended learning. Learners do well with word processing, e-mail, spreadsheets and web browsers but still lag below average in html tools. They show computer confidence at 75.3%; which gives prospects for an effective blended learning environment in regard to their computer competence and confidence. The levels of family and social support for learners stand at 61 and 75% respectively, indicating potential for blended learning to be effective. The learners’ balance between study and work is a drive factor towards blended learning effectiveness since their management of their workload vis a vis study time is at 60 and 61% of the learners are encouraged to go for study by their bosses. Learner satisfaction with the online system and its tools shows prospect for blended learning effectiveness but there are challenges in regard to locating course content and assignments, submitting their work and staying on a task during online study. Average collaborative, cognitive learning as well as learner-teacher interactions exist as important factors. Technology quality for effective blended learning is a potential for effectiveness though features like the blog and wiki are rarely used by learners. Face-to-face support is satisfactory and it should be conducted every month. There is high intrinsic motivation, satisfaction and knowledge construction as well as good performance in examinations ( M  = 62%, SD = 7.5); which indicates potentiality for blended learning effectiveness.

Significant predictors of blended learning effectiveness

Among the design features, technology quality, online tools and face-to-face support are predictors of learner satisfaction while learner characteristics of self regulation and attitudes to blended learning are predictors of satisfaction. Technology quality and interactions are the only design features predicting learner knowledge construction, while social support, among the learner backgrounds, is a predictor of knowledge construction. Self regulation as a learner characteristic is a predictor of knowledge construction. Self regulation is the only learner characteristic predicting intrinsic motivation in blended learning while technology quality, online tools and interactions are the design features predicting intrinsic motivation. However, all the independent variables are not significant predictors of learning performance in blended learning.

The high computer competences and confidence is an antecedent factor for blended learning effectiveness as noted by Hadad ( 2007 ) and this study finds learners confident and competent enough for the effectiveness of blended learning. A lack in computer skills causes failure in e-learning and blended learning as noted by Shraim and Khlaif ( 2010 ). From our study findings, this is no threat for blended learning our case as noted by our results. Contrary to Cohen et al. ( 2012 ) findings that learners’ family responsibilities and hours of employment can impede their process of learning, it is not the case here since they are drivers to the blended learning process. Time conflict, as compounded by family, employment status and management support (Packham et al., 2004 ) were noted as causes of learner failure and drop out of online courses. Our results show, on the contrary, that these factors are drivers for blended learning effectiveness because learners have a good balance between work and study and are supported by bosses to study. In agreement with Selim ( 2007 ), learner positive attitudes towards e-and blended learning environments are success factors. In line with Coldwell et al. ( 2008 ), no statistically significant differences exist between age groups. We however note that Coldwel, et al dealt with young, middle-aged and old above 45 years whereas we dealt with young and middle aged only.

Learner interactions at all levels are good enough and contrary to Astleitner, ( 2000 ) that their absence makes learners withdraw, they are a drive factor here. In line with Loukis (2007) the LMS quality, reliability and ease of use lead to learning efficiency as technology quality, online tools are predictors of learner satisfaction and intrinsic motivation. Face-to-face sessions should continue on a monthly basis as noted here and is in agreement with Marriot et al. ( 2004 ) who noted learner preference for it for facilitating social interaction and communication skills. High learner intrinsic motivation leads to persistence in online courses as noted by Menager-Beeley, ( 2004 ) and is high enough in our study. This implies a possibility of an effectiveness blended learning environment. The causes of learner dissatisfaction noted by Islam ( 2014 ) such as incompetence in the use of the LMS are contrary to our results in our study, while the one noted by Hara and Kling, ( 2001 ) as resulting from technical difficulties and ambiguous course instruction are no threat from our findings. Student-teacher interaction showed a relation with satisfaction according to Swan ( 2001 ) but is not a predictor in our study. Initiating knowledge construction by learners for blended learning effectiveness is exhibited in our findings and agrees with Rahman, Yasin and Jusof ( 2011 ). Our study has not agreed with Eom et al. ( 2006 ) who found learner interactions as predictors of learner satisfaction but agrees with Naaj et al. ( 2012 ) regarding technology as a predictor of learner satisfaction.

Conclusion and recommendations

An effective blended learning environment is necessary in undertaking innovative pedagogical approaches through the use of technology in teaching and learning. An examination of learner characteristics/background, design features and learning outcomes as factors for effectiveness can help to inform the design of effective learning environments that involve face-to-face sessions and online aspects. Most of the student characteristics and blended learning design features dealt with in this study are important factors for blended learning effectiveness. None of the independent variables were identified as significant predictors of student performance. These gaps are open for further investigation in order to understand if they can be significant predictors of blended learning effectiveness in a similar or different learning setting.

In planning to design and implement blended learning, we are mindful of the implications raised by this study which is a planning evaluation research for the design and eventual implementation of blended learning. Universities should be mindful of the interplay between the learner characteristics, design features and learning outcomes which are indicators of blended learning effectiveness. From this research, learners manifest high potential to take on blended learning more especially in regard to learner self-regulation exhibited. Blended learning is meant to increase learners’ levels of knowledge construction in order to create analytical skills in them. Learner ability to assess and critically evaluate knowledge sources is hereby established in our findings. This can go a long way in producing skilled learners who can be innovative graduates enough to satisfy employment demands through creativity and innovativeness. Technology being less of a shock to students gives potential for blended learning design. Universities and other institutions of learning should continue to emphasize blended learning approaches through installation of learning management systems along with strong internet to enable effective learning through technology especially in the developing world.

Abubakar, D. & Adetimirin. (2015). Influence of computer literacy on post-graduates’ use of e-resources in Nigerian University Libraries. Library Philosophy and Practice. From http://digitalcommons.unl.edu/libphilprac/ . Retrieved 18 Aug 2015.

Ahmad, N., & Al-Khanjari, Z. (2011). Effect of Moodle on learning: An Oman perception. International Journal of Digital Information and Wireless Communications (IJDIWC), 1 (4), 746–752.

Google Scholar  

Anderson, T. (2004). Theory and Practice of Online Learning . Canada: AU Press, Athabasca University.

Arbaugh, J. B. (2000). How classroom environment and student engagement affect learning in internet-basedMBAcourses. Business Communication Quarterly, 63 (4), 9–18.

Article   Google Scholar  

Askar, P. & Altun, A. (2008). Learner satisfaction on blended learning. E-Leader Krakow , 2008.

Astleitner, H. (2000) Dropout and distance education. A review of motivational and emotional strategies to reduce dropout in web-based distance education. In Neuwe Medien in Unterricht, Aus-und Weiterbildung Waxmann Munster, New York.

Barnard, L., Lan, W. Y., To, Y. M., Paton, V. O., & Lai, S. (2009). Measuring self regulation in online and blended learning environments’. Internet and Higher Education, 12 (1), 1–6.

Beard, L. A., Harper, C., & Riley, G. (2004). Online versus on-campus instruction: student attitudes & perceptions. TechTrends, 48 (6), 29–31.

Berenson, R., Boyles, G., & Weaver, A. (2008). Emotional intelligence as a predictor for success in online learning. International Review of Research in open & Distance Learning, 9 (2), 1–16.

Blocker, J. M., & Tucker, G. (2001). Using constructivist principles in designing and integrating online collaborative interactions. In F. Fuller & R. McBride (Eds.), Distance education. Proceedings of the Society for Information Technology & Teacher Education International Conference (pp. 32–36). ERIC Document Reproduction Service No. ED 457 822.

Cohen, K. E., Stage, F. K., Hammack, F. M., & Marcus, A. (2012). Persistence of master’s students in the United States: Developing and testing of a conceptual model . USA: PhD Dissertation, New York University.

Coldwell, J., Craig, A., Paterson, T., & Mustard, J. (2008). Online students: Relationships between participation, demographics and academic performance. The Electronic Journal of e-learning, 6 (1), 19–30.

Deci, E. L., & Ryan, R. M. (1982). Intrinsic Motivation Inventory. Available from selfdeterminationtheory.org/intrinsic-motivation-inventory/ . Accessed 2 Aug 2016.

Delone, W. H., & McLean, E. R. (2003). The Delone and McLean model of information systems success: A Ten-year update. Journal of Management Information Systems, 19 (4), 9–30.

Demirkol, M., & Kazu, I. Y. (2014). Effect of blended environment model on high school students’ academic achievement. The Turkish Online Journal of Educational Technology, 13 (1), 78–87.

Eom, S., Wen, H., & Ashill, N. (2006). The determinants of students’ perceived learning outcomes and satisfaction in university online education: an empirical investigation’. Decision Sciences Journal of Innovative Education, 4 (2), 215–235.

Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. Internet and Higher Education, 7 (2), 95–105.

Goyal, E., & Tambe, S. (2015). Effectiveness of Moodle-enabled blended learning in private Indian Business School teaching NICHE programs. The Online Journal of New Horizons in Education, 5 (2), 14–22.

Green, J., Nelson, G., Martin, A. J., & Marsh, H. (2006). The causal ordering of self-concept and academic motivation and its effect on academic achievement. International Education Journal, 7 (4), 534–546.

Guskey, T. R. (2000). Evaluating Professional Development . Thousands Oaks: Corwin Press.

Hadad, W. (2007). ICT-in-education toolkit reference handbook . InfoDev. from http://www.infodev.org/en/Publication.301.html . Retrieved 04 Aug 2015.

Hara, N. & Kling, R. (2001). Student distress in web-based distance education. Educause Quarterly. 3 (2001).

Heinich, R., Molenda, M., Russell, J. D., & Smaldino, S. E. (2001). Instructional Media and Technologies for Learning (7th ed.). Englewood Cliffs: Prentice-Hall.

Hofmann, J. (2014). Solutions to the top 10 challenges of blended learning. Top 10 challenges of blended learning. Available on cedma-europe.org .

Islam, A. K. M. N. (2014). Sources of satisfaction and dissatisfaction with a learning management system in post-adoption stage: A critical incident technique approach. Computers in Human Behaviour, 30 , 249–261.

Kelley, D. H. & Gorham, J. (2009) Effects of immediacy on recall of information. Communication Education, 37 (3), 198–207.

Kenney, J., & Newcombe, E. (2011). Adopting a blended learning approach: Challenges, encountered and lessons learned in an action research study. Journal of Asynchronous Learning Networks, 15 (1), 45–57.

Kintu, M. J., & Zhu, C. (2016). Student characteristics and learning outcomes in a blended learning environment intervention in a Ugandan University. Electronic Journal of e-Learning, 14 (3), 181–195.

Kuo, Y., Walker, A. E., Belland, B. R., & Schroder, L. E. E. (2013). A predictive study of student satisfaction in online education programs. International Review of Research in Open and Distributed Learning, 14 (1), 16–39.

Kwak, D. W., Menezes, F. M., & Sherwood, C. (2013). Assessing the impact of blended learning on student performance. Educational Technology & Society, 15 (1), 127–136.

Lim, D. H., & Kim, H. J. (2003). Motivation and learner characteristics affecting online learning and learning application. Journal of Educational Technology Systems, 31 (4), 423–439.

Lim, D. H., & Morris, M. L. (2009). Learner and instructional factors influencing learner outcomes within a blended learning environment. Educational Technology & Society, 12 (4), 282–293.

Lin, B., & Vassar, J. A. (2009). Determinants for success in online learning communities. International Journal of Web-based Communities, 5 (3), 340–350.

Loukis, E., Georgiou, S. & Pazalo, K. (2007). A value flow model for the evaluation of an e-learning service. ECIS, 2007 Proceedings, paper 175.

Lynch, R., & Dembo, M. (2004). The relationship between self regulation and online learning in a blended learning context. The International Review of Research in Open and Distributed Learning, 5 (2), 1–16.

Marriot, N., Marriot, P., & Selwyn. (2004). Accounting undergraduates’ changing use of ICT and their views on using the internet in higher education-A Research note. Accounting Education, 13 (4), 117–130.

Menager-Beeley, R. (2004). Web-based distance learning in a community college: The influence of task values on task choice, retention and commitment. (Doctoral dissertation, University of Southern California). Dissertation Abstracts International, 64 (9-A), 3191.

Naaj, M. A., Nachouki, M., & Ankit, A. (2012). Evaluating student satisfaction with blended learning in a gender-segregated environment. Journal of Information Technology Education: Research, 11 , 185–200.

Nurmela, K., Palonen, T., Lehtinen, E. & Hakkarainen, K. (2003). Developing tools for analysing CSCL process. In Wasson, B. Ludvigsen, S. & Hoppe, V. (eds), Designing for change in networked learning environments (pp 333–342). Dordrecht, The Netherlands, Kluwer.

Osgerby, J. (2013). Students’ perceptions of the introduction of a blended learning environment: An exploratory case study. Accounting Education, 22 (1), 85–99.

Oxford Group, (2013). Blended learning-current use, challenges and best practices. From http://www.kineo.com/m/0/blended-learning-report-202013.pdf . Accessed on 17 Mar 2016.

Packham, G., Jones, P., Miller, C., & Thomas, B. (2004). E-learning and retention key factors influencing student withdrawal. Education and Training, 46 (6–7), 335–342.

Pallant, J. (2010). SPSS Survival Mannual (4th ed.). Maidenhead: OUP McGraw-Hill.

Park, J.-H., & Choi, H. J. (2009). Factors influencing adult learners’ decision to drop out or persist in online learning. Educational Technology & Society, 12 (4), 207–217.

Picciano, A., & Seaman, J. (2007). K-12 online learning: A survey of U.S. school district administrators . New York, USA: Sloan-C.

Piccoli, G., Ahmad, R., & Ives, B. (2001). Web-based virtual learning environments: a research framework and a preliminary assessment of effectiveness in basic IT skill training. MIS Quarterly, 25 (4), 401–426.

Pituch, K. A., & Lee, Y. K. (2006). The influence of system characteristics on e-learning use. Computers & Education, 47 (2), 222–244.

Rahman, S. et al, (2011). Knowledge construction process in online learning. Middle East Journal of Scientific Research, 8 (2), 488–492.

Rovai, A. P. (2003). In search of higher persistence rates in distance education online programs. Computers & Education, 6 (1), 1–16.

Sankaran, S., & Bui, T. (2001). Impact of learning strategies and motivation on performance: A study in Web-based instruction. Journal of Instructional Psychology, 28 (3), 191–198.

Selim, H. M. (2007). Critical success factors for e-learning acceptance: Confirmatory factor models. Computers & Education, 49 (2), 396–413.

Shraim, K., & Khlaif, Z. N. (2010). An e-learning approach to secondary education in Palestine: opportunities and challenges. Information Technology for Development, 16 (3), 159–173.

Shrain, K. (2012). Moving towards e-learning paradigm: Readiness of higher education instructors in Palestine. International Journal on E-Learning, 11 (4), 441–463.

Song, L., Singleton, E. S., Hill, J. R., & Koh, M. H. (2004). Improving online learning: student perceptions of useful and challenging characteristics’. Internet and Higher Education, 7 (1), 59–70.

Stacey, E., & Gerbic, P. (2007). Teaching for blended learning: research perspectives from on-campus and distance students. Education and Information Technologies, 12 , 165–174.

Swan, K. (2001). Virtual interactivity: design factors affecting student satisfaction and perceived learning in asynchronous online courses. Distance Education, 22 (2), 306–331.

Article   MathSciNet   Google Scholar  

Thompson, E. (2004). Distance education drop-out: What can we do? In R. Pospisil & L. Willcoxson (Eds.), Learning Through Teaching (Proceedings of the 6th Annual Teaching Learning Forum, pp. 324–332). Perth, Australia: Murdoch University.

Tselios, N., Daskalakis, S., & Papadopoulou, M. (2011). Assessing the acceptance of a blended learning university course. Educational Technology & Society, 14 (2), 224–235.

Willging, P. A., & Johnson, S. D. (2009). Factors that influence students’ decision to drop-out of online courses. Journal of Asynchronous Learning Networks, 13 (3), 115–127.

Zhu, C. (2012). Student satisfaction, performance and knowledge construction in online collaborative learning. Educational Technology & Society, 15 (1), 127–137.

Zielinski, D. (2000). Can you keep learners online? Training, 37 (3), 64–75.

Download references

Authors’ contribution

MJK conceived the study idea, developed the conceptual framework, collected the data, analyzed it and wrote the article. CZ gave the technical advice concerning the write-up and advised on relevant corrections to be made before final submission. EK did the proof-reading of the article as well as language editing. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Author information

Authors and affiliations.

Mountains of the Moon University, P.O. Box 837, Fort Portal, Uganda

Mugenyi Justice Kintu & Edmond Kagambe

Vrije Universiteit Brussel, Pleinlaan 2, Brussels, 1050, Ixelles, Belgium

Mugenyi Justice Kintu & Chang Zhu

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mugenyi Justice Kintu .

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Kintu, M.J., Zhu, C. & Kagambe, E. Blended learning effectiveness: the relationship between student characteristics, design features and outcomes. Int J Educ Technol High Educ 14 , 7 (2017). https://doi.org/10.1186/s41239-017-0043-4

Download citation

Received : 13 July 2016

Accepted : 23 November 2016

Published : 06 February 2017

DOI : https://doi.org/10.1186/s41239-017-0043-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Blended learning effectiveness
• Learner characteristics
• Design features
• Learning outcomes and significant predictors

• Our Mission

The 10 Most Significant Education Studies of 2020

We reviewed hundreds of educational studies in 2020 and then highlighted 10 of the most significant—covering topics from virtual learning to the reading wars and the decline of standardized tests.

In the month of March of 2020, the year suddenly became a whirlwind. With a pandemic disrupting life across the entire globe, teachers scrambled to transform their physical classrooms into virtual—or even hybrid—ones, and researchers slowly began to collect insights into what works, and what doesn’t, in online learning environments around the world.

Meanwhile, neuroscientists made a convincing case for keeping handwriting in schools, and after the closure of several coal-fired power plants in Chicago, researchers reported a drop in pediatric emergency room visits and fewer absences in schools, reminding us that questions of educational equity do not begin and end at the schoolhouse door.

1. To Teach Vocabulary, Let Kids Be Thespians

When students are learning a new language, ask them to act out vocabulary words. It’s fun to unleash a child’s inner thespian, of course, but a 2020 study concluded that it also nearly doubles their ability to remember the words months later.

Researchers asked 8-year-old students to listen to words in another language and then use their hands and bodies to mimic the words—spreading their arms and pretending to fly, for example, when learning the German word flugzeug , which means “airplane.” After two months, these young actors were a remarkable 73 percent more likely to remember the new words than students who had listened without accompanying gestures. Researchers discovered similar, if slightly less dramatic, results when students looked at pictures while listening to the corresponding vocabulary. 

It’s a simple reminder that if you want students to remember something, encourage them to learn it in a variety of ways—by drawing it , acting it out, or pairing it with relevant images , for example.

2. Neuroscientists Defend the Value of Teaching Handwriting—Again

For most kids, typing just doesn’t cut it. In 2012, brain scans of preliterate children revealed crucial reading circuitry flickering to life when kids hand-printed letters and then tried to read them. The effect largely disappeared when the letters were typed or traced.

More recently, in 2020, a team of researchers studied older children—seventh graders—while they handwrote, drew, and typed words, and concluded that handwriting and drawing produced telltale neural tracings indicative of deeper learning.

“Whenever self-generated movements are included as a learning strategy, more of the brain gets stimulated,” the researchers explain, before echoing the 2012 study: “It also appears that the movements related to keyboard typing do not activate these networks the same way that drawing and handwriting do.”

It would be a mistake to replace typing with handwriting, though. All kids need to develop digital skills, and there’s evidence that technology helps children with dyslexia to overcome obstacles like note taking or illegible handwriting, ultimately freeing them to “use their time for all the things in which they are gifted,” says the Yale Center for Dyslexia and Creativity.

3. The ACT Test Just Got a Negative Score (Face Palm)

A 2020 study found that ACT test scores, which are often a key factor in college admissions, showed a weak—or even negative —relationship when it came to predicting how successful students would be in college. “There is little evidence that students will have more college success if they work to improve their ACT score,” the researchers explain, and students with very high ACT scores—but indifferent high school grades—often flamed out in college, overmatched by the rigors of a university’s academic schedule.

Just last year, the SAT—cousin to the ACT—had a similarly dubious public showing. In a major 2019 study of nearly 50,000 students led by researcher Brian Galla, and including Angela Duckworth, researchers found that high school grades were stronger predictors of four-year-college graduation than SAT scores.

The reason? Four-year high school grades, the researchers asserted, are a better indicator of crucial skills like perseverance, time management, and the ability to avoid distractions. It’s most likely those skills, in the end, that keep kids in college.

4. A Rubric Reduces Racial Grading Bias

A simple step might help undercut the pernicious effect of grading bias, a new study found: Articulate your standards clearly before you begin grading, and refer to the standards regularly during the assessment process.

In 2020, more than 1,500 teachers were recruited and asked to grade a writing sample from a fictional second-grade student. All of the sample stories were identical—but in one set, the student mentions a family member named Dashawn, while the other set references a sibling named Connor.

Teachers were 13 percent more likely to give the Connor papers a passing grade, revealing the invisible advantages that many students unknowingly benefit from. When grading criteria are vague, implicit stereotypes can insidiously “fill in the blanks,” explains the study’s author. But when teachers have an explicit set of criteria to evaluate the writing—asking whether the student “provides a well-elaborated recount of an event,” for example—the difference in grades is nearly eliminated.

5. What Do Coal-Fired Power Plants Have to Do With Learning? Plenty

When three coal-fired plants closed in the Chicago area, student absences in nearby schools dropped by 7 percent, a change largely driven by fewer emergency room visits for asthma-related problems. The stunning finding, published in a 2020 study from Duke and Penn State, underscores the role that often-overlooked environmental factors—like air quality, neighborhood crime, and noise pollution—have in keeping our children healthy and ready to learn.

At scale, the opportunity cost is staggering: About 2.3 million children in the United States still attend a public elementary or middle school located within 10 kilometers of a coal-fired plant.

The study builds on a growing body of research that reminds us that questions of educational equity do not begin and end at the schoolhouse door. What we call an achievement gap is often an equity gap, one that “takes root in the earliest years of children’s lives,” according to a 2017 study . We won’t have equal opportunity in our schools, the researchers admonish, until we are diligent about confronting inequality in our cities, our neighborhoods—and ultimately our own backyards.

6. Students Who Generate Good Questions Are Better Learners

Some of the most popular study strategies—highlighting passages, rereading notes, and underlining key sentences—are also among the least effective. A 2020 study highlighted a powerful alternative: Get students to generate questions about their learning, and gradually press them to ask more probing questions.

In the study, students who studied a topic and then generated their own questions scored an average of 14 percentage points higher on a test than students who used passive strategies like studying their notes and rereading classroom material. Creating questions, the researchers found, not only encouraged students to think more deeply about the topic but also strengthened their ability to remember what they were studying.

There are many engaging ways to have students create highly productive questions : When creating a test, you can ask students to submit their own questions, or you can use the Jeopardy! game as a platform for student-created questions.

7. Did a 2020 Study Just End the ‘Reading Wars’?

One of the most widely used reading programs was dealt a severe blow when a panel of reading experts concluded that it “would be unlikely to lead to literacy success for all of America’s public schoolchildren.”

In the 2020 study , the experts found that the controversial program—called “Units of Study” and developed over the course of four decades by Lucy Calkins at the Teachers College Reading and Writing Project—failed to explicitly and systematically teach young readers how to decode and encode written words, and was thus “in direct opposition to an enormous body of settled research.”

The study sounded the death knell for practices that de-emphasize phonics in favor of having children use multiple sources of information—like story events or illustrations—to predict the meaning of unfamiliar words, an approach often associated with “balanced literacy.” In an internal memo obtained by publisher APM, Calkins seemed to concede the point, writing that “aspects of balanced literacy need some ‘rebalancing.’”

8. A Secret to High-Performing Virtual Classrooms

In 2020, a team at Georgia State University compiled a report on virtual learning best practices. While evidence in the field is "sparse" and "inconsistent," the report noted that logistical issues like accessing materials—and not content-specific problems like failures of comprehension—were often among the most significant obstacles to online learning. It wasn’t that students didn’t understand photosynthesis in a virtual setting, in other words—it was that they didn’t find (or simply didn't access) the lesson on photosynthesis at all.

That basic insight echoed a 2019 study that highlighted the crucial need to organize virtual classrooms even more intentionally than physical ones. Remote teachers should use a single, dedicated hub for important documents like assignments; simplify communications and reminders by using one channel like email or text; and reduce visual clutter like hard-to-read fonts and unnecessary decorations throughout their virtual spaces.

Because the tools are new to everyone, regular feedback on topics like accessibility and ease of use is crucial. Teachers should post simple surveys asking questions like “Have you encountered any technical issues?” and “Can you easily locate your assignments?” to ensure that students experience a smooth-running virtual learning space.

9. Love to Learn Languages? Surprisingly, Coding May Be Right for You

Learning how to code more closely resembles learning a language such as Chinese or Spanish than learning math, a 2020 study found—upending the conventional wisdom about what makes a good programmer.

In the study, young adults with no programming experience were asked to learn Python, a popular programming language; they then took a series of tests assessing their problem-solving, math, and language skills. The researchers discovered that mathematical skill accounted for only 2 percent of a person’s ability to learn how to code, while language skills were almost nine times more predictive, accounting for 17 percent of learning ability.

That’s an important insight because all too often, programming classes require that students pass advanced math courses—a hurdle that needlessly excludes students with untapped promise, the researchers claim.

10. Researchers Cast Doubt on Reading Tasks Like ‘Finding the Main Idea’

“Content is comprehension,” declared a 2020 Fordham Institute study , sounding a note of defiance as it staked out a position in the ongoing debate over the teaching of intrinsic reading skills versus the teaching of content knowledge.

While elementary students spend an enormous amount of time working on skills like “finding the main idea” and “summarizing”—tasks born of the belief that reading is a discrete and trainable ability that transfers seamlessly across content areas—these young readers aren’t experiencing “the additional reading gains that well-intentioned educators hoped for,” the study concluded.

So what works? The researchers looked at data from more than 18,000 K–5 students, focusing on the time spent in subject areas like math, social studies, and ELA, and found that “social studies is the only subject with a clear, positive, and statistically significant effect on reading improvement.” In effect, exposing kids to rich content in civics, history, and law appeared to teach reading more effectively than our current methods of teaching reading. Perhaps defiance is no longer needed: Fordham’s conclusions are rapidly becoming conventional wisdom—and they extend beyond the limited claim of reading social studies texts. According to Natalie Wexler, the author of the well-received 2019 book  The Knowledge Gap , content knowledge and reading are intertwined. “Students with more [background] knowledge have a better chance of understanding whatever text they encounter. They’re able to retrieve more information about the topic from long-term memory, leaving more space in working memory for comprehension,” she recently told Edutopia .

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Impacts of digital technologies on education and factors influencing schools' digital capacity and transformation: A literature review

Stella timotheou.

1 CYENS Center of Excellence & Cyprus University of Technology (Cyprus Interaction Lab), Cyprus, CYENS Center of Excellence & Cyprus University of Technology, Nicosia-Limassol, Cyprus

Ourania Miliou

Yiannis dimitriadis.

2 Universidad de Valladolid (UVA), Spain, Valladolid, Spain

Sara Villagrá Sobrino

Nikoleta giannoutsou, romina cachia.

3 JRC - Joint Research Centre of the European Commission, Seville, Spain

Alejandra Martínez Monés

Andri ioannou, associated data.

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

Digital technologies have brought changes to the nature and scope of education and led education systems worldwide to adopt strategies and policies for ICT integration. The latter brought about issues regarding the quality of teaching and learning with ICTs, especially concerning the understanding, adaptation, and design of the education systems in accordance with current technological trends. These issues were emphasized during the recent COVID-19 pandemic that accelerated the use of digital technologies in education, generating questions regarding digitalization in schools. Specifically, many schools demonstrated a lack of experience and low digital capacity, which resulted in widening gaps, inequalities, and learning losses. Such results have engendered the need for schools to learn and build upon the experience to enhance their digital capacity and preparedness, increase their digitalization levels, and achieve a successful digital transformation. Given that the integration of digital technologies is a complex and continuous process that impacts different actors within the school ecosystem, there is a need to show how these impacts are interconnected and identify the factors that can encourage an effective and efficient change in the school environments. For this purpose, we conducted a non-systematic literature review. The results of the literature review were organized thematically based on the evidence presented about the impact of digital technology on education and the factors that affect the schools’ digital capacity and digital transformation. The findings suggest that ICT integration in schools impacts more than just students’ performance; it affects several other school-related aspects and stakeholders, too. Furthermore, various factors affect the impact of digital technologies on education. These factors are interconnected and play a vital role in the digital transformation process. The study results shed light on how ICTs can positively contribute to the digital transformation of schools and which factors should be considered for schools to achieve effective and efficient change.

Introduction

Digital technologies have brought changes to the nature and scope of education. Versatile and disruptive technological innovations, such as smart devices, the Internet of Things (IoT), artificial intelligence (AI), augmented reality (AR) and virtual reality (VR), blockchain, and software applications have opened up new opportunities for advancing teaching and learning (Gaol & Prasolova-Førland, 2021 ; OECD, 2021 ). Hence, in recent years, education systems worldwide have increased their investment in the integration of information and communication technology (ICT) (Fernández-Gutiérrez et al., 2020 ; Lawrence & Tar, 2018 ) and prioritized their educational agendas to adapt strategies or policies around ICT integration (European Commission, 2019 ). The latter brought about issues regarding the quality of teaching and learning with ICTs (Bates, 2015 ), especially concerning the understanding, adaptation, and design of education systems in accordance with current technological trends (Balyer & Öz, 2018 ). Studies have shown that despite the investment made in the integration of technology in schools, the results have not been promising, and the intended outcomes have not yet been achieved (Delgado et al., 2015 ; Lawrence & Tar, 2018 ). These issues were exacerbated during the COVID-19 pandemic, which forced teaching across education levels to move online (Daniel, 2020 ). Online teaching accelerated the use of digital technologies generating questions regarding the process, the nature, the extent, and the effectiveness of digitalization in schools (Cachia et al., 2021 ; König et al., 2020 ). Specifically, many schools demonstrated a lack of experience and low digital capacity, which resulted in widening gaps, inequalities, and learning losses (Blaskó et al., 2021 ; Di Pietro et al, 2020 ). Such results have engendered the need for schools to learn and build upon the experience in order to enhance their digital capacity (European Commission, 2020 ) and increase their digitalization levels (Costa et al., 2021 ). Digitalization offers possibilities for fundamental improvement in schools (OECD, 2021 ; Rott & Marouane, 2018 ) and touches many aspects of a school’s development (Delcker & Ifenthaler, 2021 ) . However, it is a complex process that requires large-scale transformative changes beyond the technical aspects of technology and infrastructure (Pettersson, 2021 ). Namely, digitalization refers to “ a series of deep and coordinated culture, workforce, and technology shifts and operating models ” (Brooks & McCormack, 2020 , p. 3) that brings cultural, organizational, and operational change through the integration of digital technologies (JISC, 2020 ). A successful digital transformation requires that schools increase their digital capacity levels, establishing the necessary “ culture, policies, infrastructure as well as digital competence of students and staff to support the effective integration of technology in teaching and learning practices ” (Costa et al, 2021 , p.163).

Given that the integration of digital technologies is a complex and continuous process that impacts different actors within the school ecosystem (Eng, 2005 ), there is a need to show how the different elements of the impact are interconnected and to identify the factors that can encourage an effective and efficient change in the school environment. To address the issues outlined above, we formulated the following research questions:

a) What is the impact of digital technologies on education?

b) Which factors might affect a school’s digital capacity and transformation?

In the present investigation, we conducted a non-systematic literature review of publications pertaining to the impact of digital technologies on education and the factors that affect a school’s digital capacity and transformation. The results of the literature review were organized thematically based on the evidence presented about the impact of digital technology on education and the factors which affect the schools’ digital capacity and digital transformation.

Methodology

The non-systematic literature review presented herein covers the main theories and research published over the past 17 years on the topic. It is based on meta-analyses and review papers found in scholarly, peer-reviewed content databases and other key studies and reports related to the concepts studied (e.g., digitalization, digital capacity) from professional and international bodies (e.g., the OECD). We searched the Scopus database, which indexes various online journals in the education sector with an international scope, to collect peer-reviewed academic papers. Furthermore, we used an all-inclusive Google Scholar search to include relevant key terms or to include studies found in the reference list of the peer-reviewed papers, and other key studies and reports related to the concepts studied by professional and international bodies. Lastly, we gathered sources from the Publications Office of the European Union ( https://op.europa.eu/en/home ); namely, documents that refer to policies related to digital transformation in education.

Regarding search terms, we first searched resources on the impact of digital technologies on education by performing the following search queries: “impact” OR “effects” AND “digital technologies” AND “education”, “impact” OR “effects” AND “ICT” AND “education”. We further refined our results by adding the terms “meta-analysis” and “review” or by adjusting the search options based on the features of each database to avoid collecting individual studies that would provide limited contributions to a particular domain. We relied on meta-analyses and review studies as these consider the findings of multiple studies to offer a more comprehensive view of the research in a given area (Schuele & Justice, 2006 ). Specifically, meta-analysis studies provided quantitative evidence based on statistically verifiable results regarding the impact of educational interventions that integrate digital technologies in school classrooms (Higgins et al., 2012 ; Tolani-Brown et al., 2011 ).

However, quantitative data does not offer explanations for the challenges or difficulties experienced during ICT integration in learning and teaching (Tolani-Brown et al., 2011 ). To fill this gap, we analyzed literature reviews and gathered in-depth qualitative evidence of the benefits and implications of technology integration in schools. In the analysis presented herein, we also included policy documents and reports from professional and international bodies and governmental reports, which offered useful explanations of the key concepts of this study and provided recent evidence on digital capacity and transformation in education along with policy recommendations. The inclusion and exclusion criteria that were considered in this study are presented in Table ​ Table1 1 .

Inclusion and exclusion criteria for the selection of resources on the impact of digital technologies on education

To ensure a reliable extraction of information from each study and assist the research synthesis we selected the study characteristics of interest (impact) and constructed coding forms. First, an overview of the synthesis was provided by the principal investigator who described the processes of coding, data entry, and data management. The coders followed the same set of instructions but worked independently. To ensure a common understanding of the process between coders, a sample of ten studies was tested. The results were compared, and the discrepancies were identified and resolved. Additionally, to ensure an efficient coding process, all coders participated in group meetings to discuss additions, deletions, and modifications (Stock, 1994 ). Due to the methodological diversity of the studied documents we began to synthesize the literature review findings based on similar study designs. Specifically, most of the meta-analysis studies were grouped in one category due to the quantitative nature of the measured impact. These studies tended to refer to student achievement (Hattie et al., 2014 ). Then, we organized the themes of the qualitative studies in several impact categories. Lastly, we synthesized both review and meta-analysis data across the categories. In order to establish a collective understanding of the concept of impact, we referred to a previous impact study by Balanskat ( 2009 ) which investigated the impact of technology in primary schools. In this context, the impact had a more specific ICT-related meaning and was described as “ a significant influence or effect of ICT on the measured or perceived quality of (parts of) education ” (Balanskat, 2009 , p. 9). In the study presented herein, the main impacts are in relation to learning and learners, teaching, and teachers, as well as other key stakeholders who are directly or indirectly connected to the school unit.

The study’s results identified multiple dimensions of the impact of digital technologies on students’ knowledge, skills, and attitudes; on equality, inclusion, and social integration; on teachers’ professional and teaching practices; and on other school-related aspects and stakeholders. The data analysis indicated various factors that might affect the schools’ digital capacity and transformation, such as digital competencies, the teachers’ personal characteristics and professional development, as well as the school’s leadership and management, administration, infrastructure, etc. The impacts and factors found in the literature review are presented below.

Impacts of digital technologies on students’ knowledge, skills, attitudes, and emotions

The impact of ICT use on students’ knowledge, skills, and attitudes has been investigated early in the literature. Eng ( 2005 ) found a small positive effect between ICT use and students' learning. Specifically, the author reported that access to computer-assisted instruction (CAI) programs in simulation or tutorial modes—used to supplement rather than substitute instruction – could enhance student learning. The author reported studies showing that teachers acknowledged the benefits of ICT on pupils with special educational needs; however, the impact of ICT on students' attainment was unclear. Balanskat et al. ( 2006 ) found a statistically significant positive association between ICT use and higher student achievement in primary and secondary education. The authors also reported improvements in the performance of low-achieving pupils. The use of ICT resulted in further positive gains for students, namely increased attention, engagement, motivation, communication and process skills, teamwork, and gains related to their behaviour towards learning. Evidence from qualitative studies showed that teachers, students, and parents recognized the positive impact of ICT on students' learning regardless of their competence level (strong/weak students). Punie et al. ( 2006 ) documented studies that showed positive results of ICT-based learning for supporting low-achieving pupils and young people with complex lives outside the education system. Liao et al. ( 2007 ) reported moderate positive effects of computer application instruction (CAI, computer simulations, and web-based learning) over traditional instruction on primary school student's achievement. Similarly, Tamim et al. ( 2011 ) reported small to moderate positive effects between the use of computer technology (CAI, ICT, simulations, computer-based instruction, digital and hypermedia) and student achievement in formal face-to-face classrooms compared to classrooms that did not use technology. Jewitt et al., ( 2011 ) found that the use of learning platforms (LPs) (virtual learning environments, management information systems, communication technologies, and information- and resource-sharing technologies) in schools allowed primary and secondary students to access a wider variety of quality learning resources, engage in independent and personalized learning, and conduct self- and peer-review; LPs also provide opportunities for teacher assessment and feedback. Similar findings were reported by Fu ( 2013 ), who documented a list of benefits and opportunities of ICT use. According to the author, the use of ICTs helps students access digital information and course content effectively and efficiently, supports student-centered and self-directed learning, as well as the development of a creative learning environment where more opportunities for critical thinking skills are offered, and promotes collaborative learning in a distance-learning environment. Higgins et al. ( 2012 ) found consistent but small positive associations between the use of technology and learning outcomes of school-age learners (5–18-year-olds) in studies linking the provision and use of technology with attainment. Additionally, Chauhan ( 2017 ) reported a medium positive effect of technology on the learning effectiveness of primary school students compared to students who followed traditional learning instruction.

The rise of mobile technologies and hardware devices instigated investigations into their impact on teaching and learning. Sung et al. ( 2016 ) reported a moderate effect on students' performance from the use of mobile devices in the classroom compared to the use of desktop computers or the non-use of mobile devices. Schmid et al. ( 2014 ) reported medium–low to low positive effects of technology integration (e.g., CAI, ICTs) in the classroom on students' achievement and attitude compared to not using technology or using technology to varying degrees. Tamim et al. ( 2015 ) found a low statistically significant effect of the use of tablets and other smart devices in educational contexts on students' achievement outcomes. The authors suggested that tablets offered additional advantages to students; namely, they reported improvements in students’ notetaking, organizational and communication skills, and creativity. Zheng et al. ( 2016 ) reported a small positive effect of one-to-one laptop programs on students’ academic achievement across subject areas. Additional reported benefits included student-centered, individualized, and project-based learning enhanced learner engagement and enthusiasm. Additionally, the authors found that students using one-to-one laptop programs tended to use technology more frequently than in non-laptop classrooms, and as a result, they developed a range of skills (e.g., information skills, media skills, technology skills, organizational skills). Haßler et al. ( 2016 ) found that most interventions that included the use of tablets across the curriculum reported positive learning outcomes. However, from 23 studies, five reported no differences, and two reported a negative effect on students' learning outcomes. Similar results were indicated by Kalati and Kim ( 2022 ) who investigated the effect of touchscreen technologies on young students’ learning. Specifically, from 53 studies, 34 advocated positive effects of touchscreen devices on children’s learning, 17 obtained mixed findings and two studies reported negative effects.

More recently, approaches that refer to the impact of gamification with the use of digital technologies on teaching and learning were also explored. A review by Pan et al. ( 2022 ) that examined the role of learning games in fostering mathematics education in K-12 settings, reported that gameplay improved students’ performance. Integration of digital games in teaching was also found as a promising pedagogical practice in STEM education that could lead to increased learning gains (Martinez et al., 2022 ; Wang et al., 2022 ). However, although Talan et al. ( 2020 ) reported a medium effect of the use of educational games (both digital and non-digital) on academic achievement, the effect of non-digital games was higher.

Over the last two years, the effects of more advanced technologies on teaching and learning were also investigated. Garzón and Acevedo ( 2019 ) found that AR applications had a medium effect on students' learning outcomes compared to traditional lectures. Similarly, Garzón et al. ( 2020 ) showed that AR had a medium impact on students' learning gains. VR applications integrated into various subjects were also found to have a moderate effect on students’ learning compared to control conditions (traditional classes, e.g., lectures, textbooks, and multimedia use, e.g., images, videos, animation, CAI) (Chen et al., 2022b ). Villena-Taranilla et al. ( 2022 ) noted the moderate effect of VR technologies on students’ learning when these were applied in STEM disciplines. In the same meta-analysis, Villena-Taranilla et al. ( 2022 ) highlighted the role of immersive VR, since its effect on students’ learning was greater (at a high level) across educational levels (K-6) compared to semi-immersive and non-immersive integrations. In another meta-analysis study, the effect size of the immersive VR was small and significantly differentiated across educational levels (Coban et al., 2022 ). The impact of AI on education was investigated by Su and Yang ( 2022 ) and Su et al. ( 2022 ), who showed that this technology significantly improved students’ understanding of AI computer science and machine learning concepts.

It is worth noting that the vast majority of studies referred to learning gains in specific subjects. Specifically, several studies examined the impact of digital technologies on students’ literacy skills and reported positive effects on language learning (Balanskat et al., 2006 ; Grgurović et al., 2013 ; Friedel et al., 2013 ; Zheng et al., 2016 ; Chen et al., 2022b ; Savva et al., 2022 ). Also, several studies documented positive effects on specific language learning areas, namely foreign language learning (Kao, 2014 ), writing (Higgins et al., 2012 ; Wen & Walters, 2022 ; Zheng et al., 2016 ), as well as reading and comprehension (Cheung & Slavin, 2011 ; Liao et al., 2007 ; Schwabe et al., 2022 ). ICTs were also found to have a positive impact on students' performance in STEM (science, technology, engineering, and mathematics) disciplines (Arztmann et al., 2022 ; Bado, 2022 ; Villena-Taranilla et al., 2022 ; Wang et al., 2022 ). Specifically, a number of studies reported positive impacts on students’ achievement in mathematics (Balanskat et al., 2006 ; Hillmayr et al., 2020 ; Li & Ma, 2010 ; Pan et al., 2022 ; Ran et al., 2022 ; Verschaffel et al., 2019 ; Zheng et al., 2016 ). Furthermore, studies documented positive effects of ICTs on science learning (Balanskat et al., 2006 ; Liao et al., 2007 ; Zheng et al., 2016 ; Hillmayr et al., 2020 ; Kalemkuş & Kalemkuş, 2022 ; Lei et al., 2022a ). Çelik ( 2022 ) also noted that computer simulations can help students understand learning concepts related to science. Furthermore, some studies documented that the use of ICTs had a positive impact on students’ achievement in other subjects, such as geography, history, music, and arts (Chauhan, 2017 ; Condie & Munro, 2007 ), and design and technology (Balanskat et al., 2006 ).

More specific positive learning gains were reported in a number of skills, e.g., problem-solving skills and pattern exploration skills (Higgins et al., 2012 ), metacognitive learning outcomes (Verschaffel et al., 2019 ), literacy skills, computational thinking skills, emotion control skills, and collaborative inquiry skills (Lu et al., 2022 ; Su & Yang, 2022 ; Su et al., 2022 ). Additionally, several investigations have reported benefits from the use of ICT on students’ creativity (Fielding & Murcia, 2022 ; Liu et al., 2022 ; Quah & Ng, 2022 ). Lastly, digital technologies were also found to be beneficial for enhancing students’ lifelong learning skills (Haleem et al., 2022 ).

Apart from gaining knowledge and skills, studies also reported improvement in motivation and interest in mathematics (Higgins et. al., 2019 ; Fadda et al., 2022 ) and increased positive achievement emotions towards several subjects during interventions using educational games (Lei et al., 2022a ). Chen et al. ( 2022a ) also reported a small but positive effect of digital health approaches in bullying and cyberbullying interventions with K-12 students, demonstrating that technology-based approaches can help reduce bullying and related consequences by providing emotional support, empowerment, and change of attitude. In their meta-review study, Su et al. ( 2022 ) also documented that AI technologies effectively strengthened students’ attitudes towards learning. In another meta-analysis, Arztmann et al. ( 2022 ) reported positive effects of digital games on motivation and behaviour towards STEM subjects.

Impacts of digital technologies on equality, inclusion and social integration

Although most of the reviewed studies focused on the impact of ICTs on students’ knowledge, skills, and attitudes, reports were also made on other aspects in the school context, such as equality, inclusion, and social integration. Condie and Munro ( 2007 ) documented research interventions investigating how ICT can support pupils with additional or special educational needs. While those interventions were relatively small scale and mostly based on qualitative data, their findings indicated that the use of ICTs enabled the development of communication, participation, and self-esteem. A recent meta-analysis (Baragash et al., 2022 ) with 119 participants with different disabilities, reported a significant overall effect size of AR on their functional skills acquisition. Koh’s meta-analysis ( 2022 ) also revealed that students with intellectual and developmental disabilities improved their competence and performance when they used digital games in the lessons.

Istenic Starcic and Bagon ( 2014 ) found that the role of ICT in inclusion and the design of pedagogical and technological interventions was not sufficiently explored in educational interventions with people with special needs; however, some benefits of ICT use were found in students’ social integration. The issue of gender and technology use was mentioned in a small number of studies. Zheng et al. ( 2016 ) reported a statistically significant positive interaction between one-to-one laptop programs and gender. Specifically, the results showed that girls and boys alike benefitted from the laptop program, but the effect on girls’ achievement was smaller than that on boys’. Along the same lines, Arztmann et al. ( 2022 ) reported no difference in the impact of game-based learning between boys and girls, arguing that boys and girls equally benefited from game-based interventions in STEM domains. However, results from a systematic review by Cussó-Calabuig et al. ( 2018 ) found limited and low-quality evidence on the effects of intensive use of computers on gender differences in computer anxiety, self-efficacy, and self-confidence. Based on their view, intensive use of computers can reduce gender differences in some areas and not in others, depending on contextual and implementation factors.

Impacts of digital technologies on teachers’ professional and teaching practices

Various research studies have explored the impact of ICT on teachers’ instructional practices and student assessment. Friedel et al. ( 2013 ) found that the use of mobile devices by students enabled teachers to successfully deliver content (e.g., mobile serious games), provide scaffolding, and facilitate synchronous collaborative learning. The integration of digital games in teaching and learning activities also gave teachers the opportunity to study and apply various pedagogical practices (Bado, 2022 ). Specifically, Bado ( 2022 ) found that teachers who implemented instructional activities in three stages (pre-game, game, and post-game) maximized students’ learning outcomes and engagement. For instance, during the pre-game stage, teachers focused on lectures and gameplay training, at the game stage teachers provided scaffolding on content, addressed technical issues, and managed the classroom activities. During the post-game stage, teachers organized activities for debriefing to ensure that the gameplay had indeed enhanced students’ learning outcomes.

Furthermore, ICT can increase efficiency in lesson planning and preparation by offering possibilities for a more collaborative approach among teachers. The sharing of curriculum plans and the analysis of students’ data led to clearer target settings and improvements in reporting to parents (Balanskat et al., 2006 ).

Additionally, the use and application of digital technologies in teaching and learning were found to enhance teachers’ digital competence. Balanskat et al. ( 2006 ) documented studies that revealed that the use of digital technologies in education had a positive effect on teachers’ basic ICT skills. The greatest impact was found on teachers with enough experience in integrating ICTs in their teaching and/or who had recently participated in development courses for the pedagogical use of technologies in teaching. Punie et al. ( 2006 ) reported that the provision of fully equipped multimedia portable computers and the development of online teacher communities had positive impacts on teachers’ confidence and competence in the use of ICTs.

Moreover, online assessment via ICTs benefits instruction. In particular, online assessments support the digitalization of students’ work and related logistics, allow teachers to gather immediate feedback and readjust to new objectives, and support the improvement of the technical quality of tests by providing more accurate results. Additionally, the capabilities of ICTs (e.g., interactive media, simulations) create new potential methods of testing specific skills, such as problem-solving and problem-processing skills, meta-cognitive skills, creativity and communication skills, and the ability to work productively in groups (Punie et al., 2006 ).

Impacts of digital technologies on other school-related aspects and stakeholders

There is evidence that the effective use of ICTs and the data transmission offered by broadband connections help improve administration (Balanskat et al., 2006 ). Specifically, ICTs have been found to provide better management systems to schools that have data gathering procedures in place. Condie and Munro ( 2007 ) reported impacts from the use of ICTs in schools in the following areas: attendance monitoring, assessment records, reporting to parents, financial management, creation of repositories for learning resources, and sharing of information amongst staff. Such data can be used strategically for self-evaluation and monitoring purposes which in turn can result in school improvements. Additionally, they reported that online access to other people with similar roles helped to reduce headteachers’ isolation by offering them opportunities to share insights into the use of ICT in learning and teaching and how it could be used to support school improvement. Furthermore, ICTs provided more efficient and successful examination management procedures, namely less time-consuming reporting processes compared to paper-based examinations and smooth communications between schools and examination authorities through electronic data exchange (Punie et al., 2006 ).

Zheng et al. ( 2016 ) reported that the use of ICTs improved home-school relationships. Additionally, Escueta et al. ( 2017 ) reported several ICT programs that had improved the flow of information from the school to parents. Particularly, they documented that the use of ICTs (learning management systems, emails, dedicated websites, mobile phones) allowed for personalized and customized information exchange between schools and parents, such as attendance records, upcoming class assignments, school events, and students’ grades, which generated positive results on students’ learning outcomes and attainment. Such information exchange between schools and families prompted parents to encourage their children to put more effort into their schoolwork.

The above findings suggest that the impact of ICT integration in schools goes beyond students’ performance in school subjects. Specifically, it affects a number of school-related aspects, such as equality and social integration, professional and teaching practices, and diverse stakeholders. In Table ​ Table2, 2 , we summarize the different impacts of digital technologies on school stakeholders based on the literature review, while in Table ​ Table3 3 we organized the tools/platforms and practices/policies addressed in the meta-analyses, literature reviews, EU reports, and international bodies included in the manuscript.

The impact of digital technologies on schools’ stakeholders based on the literature review

Tools/platforms and practices/policies addressed in the meta-analyses, literature reviews, EU reports, and international bodies included in the manuscript

Additionally, based on the results of the literature review, there are many types of digital technologies with different affordances (see, for example, studies on VR vs Immersive VR), which evolve over time (e.g. starting from CAIs in 2005 to Augmented and Virtual reality 2020). Furthermore, these technologies are linked to different pedagogies and policy initiatives, which are critical factors in the study of impact. Table ​ Table3 3 summarizes the different tools and practices that have been used to examine the impact of digital technologies on education since 2005 based on the review results.

Factors that affect the integration of digital technologies

Although the analysis of the literature review demonstrated different impacts of the use of digital technology on education, several authors highlighted the importance of various factors, besides the technology itself, that affect this impact. For example, Liao et al. ( 2007 ) suggested that future studies should carefully investigate which factors contribute to positive outcomes by clarifying the exact relationship between computer applications and learning. Additionally, Haßler et al., ( 2016 ) suggested that the neutral findings regarding the impact of tablets on students learning outcomes in some of the studies included in their review should encourage educators, school leaders, and school officials to further investigate the potential of such devices in teaching and learning. Several other researchers suggested that a number of variables play a significant role in the impact of ICTs on students’ learning that could be attributed to the school context, teaching practices and professional development, the curriculum, and learners’ characteristics (Underwood, 2009 ; Tamim et al., 2011 ; Higgins et al., 2012 ; Archer et al., 2014 ; Sung et al., 2016 ; Haßler et al., 2016 ; Chauhan, 2017 ; Lee et al., 2020 ; Tang et al., 2022 ).

Digital competencies

One of the most common challenges reported in studies that utilized digital tools in the classroom was the lack of students’ skills on how to use them. Fu ( 2013 ) found that students’ lack of technical skills is a barrier to the effective use of ICT in the classroom. Tamim et al. ( 2015 ) reported that students faced challenges when using tablets and smart mobile devices, associated with the technical issues or expertise needed for their use and the distracting nature of the devices and highlighted the need for teachers’ professional development. Higgins et al. ( 2012 ) reported that skills training about the use of digital technologies is essential for learners to fully exploit the benefits of instruction.

Delgado et al. ( 2015 ), meanwhile, reported studies that showed a strong positive association between teachers’ computer skills and students’ use of computers. Teachers’ lack of ICT skills and familiarization with technologies can become a constraint to the effective use of technology in the classroom (Balanskat et al., 2006 ; Delgado et al., 2015 ).

It is worth noting that the way teachers are introduced to ICTs affects the impact of digital technologies on education. Previous studies have shown that teachers may avoid using digital technologies due to limited digital skills (Balanskat, 2006 ), or they prefer applying “safe” technologies, namely technologies that their own teachers used and with which they are familiar (Condie & Munro, 2007 ). In this regard, the provision of digital skills training and exposure to new digital tools might encourage teachers to apply various technologies in their lessons (Condie & Munro, 2007 ). Apart from digital competence, technical support in the school setting has also been shown to affect teachers’ use of technology in their classrooms (Delgado et al., 2015 ). Ferrari et al. ( 2011 ) found that while teachers’ use of ICT is high, 75% stated that they needed more institutional support and a shift in the mindset of educational actors to achieve more innovative teaching practices. The provision of support can reduce time and effort as well as cognitive constraints, which could cause limited ICT integration in the school lessons by teachers (Escueta et al., 2017 ).

Teachers’ personal characteristics, training approaches, and professional development

Teachers’ personal characteristics and professional development affect the impact of digital technologies on education. Specifically, Cheok and Wong ( 2015 ) found that teachers’ personal characteristics (e.g., anxiety, self-efficacy) are associated with their satisfaction and engagement with technology. Bingimlas ( 2009 ) reported that lack of confidence, resistance to change, and negative attitudes in using new technologies in teaching are significant determinants of teachers’ levels of engagement in ICT. The same author reported that the provision of technical support, motivation support (e.g., awards, sufficient time for planning), and training on how technologies can benefit teaching and learning can eliminate the above barriers to ICT integration. Archer et al. ( 2014 ) found that comfort levels in using technology are an important predictor of technology integration and argued that it is essential to provide teachers with appropriate training and ongoing support until they are comfortable with using ICTs in the classroom. Hillmayr et al. ( 2020 ) documented that training teachers on ICT had an important effecton students’ learning.

According to Balanskat et al. ( 2006 ), the impact of ICTs on students’ learning is highly dependent on the teachers’ capacity to efficiently exploit their application for pedagogical purposes. Results obtained from the Teaching and Learning International Survey (TALIS) (OECD, 2021 ) revealed that although schools are open to innovative practices and have the capacity to adopt them, only 39% of teachers in the European Union reported that they are well or very well prepared to use digital technologies for teaching. Li and Ma ( 2010 ) and Hardman ( 2019 ) showed that the positive effect of technology on students’ achievement depends on the pedagogical practices used by teachers. Schmid et al. ( 2014 ) reported that learning was best supported when students were engaged in active, meaningful activities with the use of technological tools that provided cognitive support. Tamim et al. ( 2015 ) compared two different pedagogical uses of tablets and found a significant moderate effect when the devices were used in a student-centered context and approach rather than within teacher-led environments. Similarly, Garzón and Acevedo ( 2019 ) and Garzón et al. ( 2020 ) reported that the positive results from the integration of AR applications could be attributed to the existence of different variables which could influence AR interventions (e.g., pedagogical approach, learning environment, and duration of the intervention). Additionally, Garzón et al. ( 2020 ) suggested that the pedagogical resources that teachers used to complement their lectures and the pedagogical approaches they applied were crucial to the effective integration of AR on students’ learning gains. Garzón and Acevedo ( 2019 ) also emphasized that the success of a technology-enhanced intervention is based on both the technology per se and its characteristics and on the pedagogical strategies teachers choose to implement. For instance, their results indicated that the collaborative learning approach had the highest impact on students’ learning gains among other approaches (e.g., inquiry-based learning, situated learning, or project-based learning). Ran et al. ( 2022 ) also found that the use of technology to design collaborative and communicative environments showed the largest moderator effects among the other approaches.

Hattie ( 2008 ) reported that the effective use of computers is associated with training teachers in using computers as a teaching and learning tool. Zheng et al. ( 2016 ) noted that in addition to the strategies teachers adopt in teaching, ongoing professional development is also vital in ensuring the success of technology implementation programs. Sung et al. ( 2016 ) found that research on the use of mobile devices to support learning tends to report that the insufficient preparation of teachers is a major obstacle in implementing effective mobile learning programs in schools. Friedel et al. ( 2013 ) found that providing training and support to teachers increased the positive impact of the interventions on students’ learning gains. Trucano ( 2005 ) argued that positive impacts occur when digital technologies are used to enhance teachers’ existing pedagogical philosophies. Higgins et al. ( 2012 ) found that the types of technologies used and how they are used could also affect students’ learning. The authors suggested that training and professional development of teachers that focuses on the effective pedagogical use of technology to support teaching and learning is an important component of successful instructional approaches (Higgins et al., 2012 ). Archer et al. ( 2014 ) found that studies that reported ICT interventions during which teachers received training and support had moderate positive effects on students’ learning outcomes, which were significantly higher than studies where little or no detail about training and support was mentioned. Fu ( 2013 ) reported that the lack of teachers’ knowledge and skills on the technical and instructional aspects of ICT use in the classroom, in-service training, pedagogy support, technical and financial support, as well as the lack of teachers’ motivation and encouragement to integrate ICT on their teaching were significant barriers to the integration of ICT in education.

School leadership and management

Management and leadership are important cornerstones in the digital transformation process (Pihir et al., 2018 ). Zheng et al. ( 2016 ) documented leadership among the factors positively affecting the successful implementation of technology integration in schools. Strong leadership, strategic planning, and systematic integration of digital technologies are prerequisites for the digital transformation of education systems (Ređep, 2021 ). Management and leadership play a significant role in formulating policies that are translated into practice and ensure that developments in ICT become embedded into the life of the school and in the experiences of staff and pupils (Condie & Munro, 2007 ). Policy support and leadership must include the provision of an overall vision for the use of digital technologies in education, guidance for students and parents, logistical support, as well as teacher training (Conrads et al., 2017 ). Unless there is a commitment throughout the school, with accountability for progress at key points, it is unlikely for ICT integration to be sustained or become part of the culture (Condie & Munro, 2007 ). To achieve this, principals need to adopt and promote a whole-institution strategy and build a strong mutual support system that enables the school’s technological maturity (European Commission, 2019 ). In this context, school culture plays an essential role in shaping the mindsets and beliefs of school actors towards successful technology integration. Condie and Munro ( 2007 ) emphasized the importance of the principal’s enthusiasm and work as a source of inspiration for the school staff and the students to cultivate a culture of innovation and establish sustainable digital change. Specifically, school leaders need to create conditions in which the school staff is empowered to experiment and take risks with technology (Elkordy & Lovinelli, 2020 ).

In order for leaders to achieve the above, it is important to develop capacities for learning and leading, advocating professional learning, and creating support systems and structures (European Commission, 2019 ). Digital technology integration in education systems can be challenging and leadership needs guidance to achieve it. Such guidance can be introduced through the adoption of new methods and techniques in strategic planning for the integration of digital technologies (Ređep, 2021 ). Even though the role of leaders is vital, the relevant training offered to them has so far been inadequate. Specifically, only a third of the education systems in Europe have put in place national strategies that explicitly refer to the training of school principals (European Commission, 2019 , p. 16).

Connectivity, infrastructure, and government and other support

The effective integration of digital technologies across levels of education presupposes the development of infrastructure, the provision of digital content, and the selection of proper resources (Voogt et al., 2013 ). Particularly, a high-quality broadband connection in the school increases the quality and quantity of educational activities. There is evidence that ICT increases and formalizes cooperative planning between teachers and cooperation with managers, which in turn has a positive impact on teaching practices (Balanskat et al., 2006 ). Additionally, ICT resources, including software and hardware, increase the likelihood of teachers integrating technology into the curriculum to enhance their teaching practices (Delgado et al., 2015 ). For example, Zheng et al. ( 2016 ) found that the use of one-on-one laptop programs resulted in positive changes in teaching and learning, which would not have been accomplished without the infrastructure and technical support provided to teachers. Delgado et al. ( 2015 ) reported that limited access to technology (insufficient computers, peripherals, and software) and lack of technical support are important barriers to ICT integration. Access to infrastructure refers not only to the availability of technology in a school but also to the provision of a proper amount and the right types of technology in locations where teachers and students can use them. Effective technical support is a central element of the whole-school strategy for ICT (Underwood, 2009 ). Bingimlas ( 2009 ) reported that lack of technical support in the classroom and whole-school resources (e.g., failing to connect to the Internet, printers not printing, malfunctioning computers, and working on old computers) are significant barriers that discourage the use of ICT by teachers. Moreover, poor quality and inadequate hardware maintenance, and unsuitable educational software may discourage teachers from using ICTs (Balanskat et al., 2006 ; Bingimlas, 2009 ).

Government support can also impact the integration of ICTs in teaching. Specifically, Balanskat et al. ( 2006 ) reported that government interventions and training programs increased teachers’ enthusiasm and positive attitudes towards ICT and led to the routine use of embedded ICT.

Lastly, another important factor affecting digital transformation is the development and quality assurance of digital learning resources. Such resources can be support textbooks and related materials or resources that focus on specific subjects or parts of the curriculum. Policies on the provision of digital learning resources are essential for schools and can be achieved through various actions. For example, some countries are financing web portals that become repositories, enabling teachers to share resources or create their own. Additionally, they may offer e-learning opportunities or other services linked to digital education. In other cases, specific agencies of projects have also been set up to develop digital resources (Eurydice, 2019 ).

Administration and digital data management

The digital transformation of schools involves organizational improvements at the level of internal workflows, communication between the different stakeholders, and potential for collaboration. Vuorikari et al. ( 2020 ) presented evidence that digital technologies supported the automation of administrative practices in schools and reduced the administration’s workload. There is evidence that digital data affects the production of knowledge about schools and has the power to transform how schooling takes place. Specifically, Sellar ( 2015 ) reported that data infrastructure in education is developing due to the demand for “ information about student outcomes, teacher quality, school performance, and adult skills, associated with policy efforts to increase human capital and productivity practices ” (p. 771). In this regard, practices, such as datafication which refers to the “ translation of information about all kinds of things and processes into quantified formats” have become essential for decision-making based on accountability reports about the school’s quality. The data could be turned into deep insights about education or training incorporating ICTs. For example, measuring students’ online engagement with the learning material and drawing meaningful conclusions can allow teachers to improve their educational interventions (Vuorikari et al., 2020 ).

Students’ socioeconomic background and family support

Research show that the active engagement of parents in the school and their support for the school’s work can make a difference to their children’s attitudes towards learning and, as a result, their achievement (Hattie, 2008 ). In recent years, digital technologies have been used for more effective communication between school and family (Escueta et al., 2017 ). The European Commission ( 2020 ) presented data from a Eurostat survey regarding the use of computers by students during the pandemic. The data showed that younger pupils needed additional support and guidance from parents and the challenges were greater for families in which parents had lower levels of education and little to no digital skills.

In this regard, the socio-economic background of the learners and their socio-cultural environment also affect educational achievements (Punie et al., 2006 ). Trucano documented that the use of computers at home positively influenced students’ confidence and resulted in more frequent use at school, compared to students who had no home access (Trucano, 2005 ). In this sense, the socio-economic background affects the access to computers at home (OECD, 2015 ) which in turn influences the experience of ICT, an important factor for school achievement (Punie et al., 2006 ; Underwood, 2009 ). Furthermore, parents from different socio-economic backgrounds may have different abilities and availability to support their children in their learning process (Di Pietro et al., 2020 ).

Schools’ socioeconomic context and emergency situations

The socio-economic context of the school is closely related to a school’s digital transformation. For example, schools in disadvantaged, rural, or deprived areas are likely to lack the digital capacity and infrastructure required to adapt to the use of digital technologies during emergency periods, such as the COVID-19 pandemic (Di Pietro et al., 2020 ). Data collected from school principals confirmed that in several countries, there is a rural/urban divide in connectivity (OECD, 2015 ).

Emergency periods also affect the digitalization of schools. The COVID-19 pandemic led to the closure of schools and forced them to seek appropriate and connective ways to keep working on the curriculum (Di Pietro et al., 2020 ). The sudden large-scale shift to distance and online teaching and learning also presented challenges around quality and equity in education, such as the risk of increased inequalities in learning, digital, and social, as well as teachers facing difficulties coping with this demanding situation (European Commission, 2020 ).

Looking at the findings of the above studies, we can conclude that the impact of digital technologies on education is influenced by various actors and touches many aspects of the school ecosystem. Figure  1 summarizes the factors affecting the digital technologies’ impact on school stakeholders based on the findings from the literature review.

Factors that affect the impact of ICTs on education

The findings revealed that the use of digital technologies in education affects a variety of actors within a school’s ecosystem. First, we observed that as technologies evolve, so does the interest of the research community to apply them to school settings. Figure  2 summarizes the trends identified in current research around the impact of digital technologies on schools’ digital capacity and transformation as found in the present study. Starting as early as 2005, when computers, simulations, and interactive boards were the most commonly applied tools in school interventions (e.g., Eng, 2005 ; Liao et al., 2007 ; Moran et al., 2008 ; Tamim et al., 2011 ), moving towards the use of learning platforms (Jewitt et al., 2011 ), then to the use of mobile devices and digital games (e.g., Tamim et al., 2015 ; Sung et al., 2016 ; Talan et al., 2020 ), as well as e-books (e.g., Savva et al., 2022 ), to the more recent advanced technologies, such as AR and VR applications (e.g., Garzón & Acevedo, 2019 ; Garzón et al., 2020 ; Kalemkuş & Kalemkuş, 2022 ), or robotics and AI (e.g., Su & Yang, 2022 ; Su et al., 2022 ). As this evolution shows, digital technologies are a concept in flux with different affordances and characteristics. Additionally, from an instructional perspective, there has been a growing interest in different modes and models of content delivery such as online, blended, and hybrid modes (e.g., Cheok & Wong, 2015 ; Kazu & Yalçin, 2022 ; Ulum, 2022 ). This is an indication that the value of technologies to support teaching and learning as well as other school-related practices is increasingly recognized by the research and school community. The impact results from the literature review indicate that ICT integration on students’ learning outcomes has effects that are small (Coban et al., 2022 ; Eng, 2005 ; Higgins et al., 2012 ; Schmid et al., 2014 ; Tamim et al., 2015 ; Zheng et al., 2016 ) to moderate (Garzón & Acevedo, 2019 ; Garzón et al., 2020 ; Liao et al., 2007 ; Sung et al., 2016 ; Talan et al., 2020 ; Wen & Walters, 2022 ). That said, a number of recent studies have reported high effect sizes (e.g., Kazu & Yalçin, 2022 ).

Current work and trends in the study of the impact of digital technologies on schools’ digital capacity

Based on these findings, several authors have suggested that the impact of technology on education depends on several variables and not on the technology per se (Tamim et al., 2011 ; Higgins et al., 2012 ; Archer et al., 2014 ; Sung et al., 2016 ; Haßler et al., 2016 ; Chauhan, 2017 ; Lee et al., 2020 ; Lei et al., 2022a ). While the impact of ICTs on student achievement has been thoroughly investigated by researchers, other aspects related to school life that are also affected by ICTs, such as equality, inclusion, and social integration have received less attention. Further analysis of the literature review has revealed a greater investment in ICT interventions to support learning and teaching in the core subjects of literacy and STEM disciplines, especially mathematics, and science. These were the most common subjects studied in the reviewed papers often drawing on national testing results, while studies that investigated other subject areas, such as social studies, were limited (Chauhan, 2017 ; Condie & Munro, 2007 ). As such, research is still lacking impact studies that focus on the effects of ICTs on a range of curriculum subjects.

The qualitative research provided additional information about the impact of digital technologies on education, documenting positive effects and giving more details about implications, recommendations, and future research directions. Specifically, the findings regarding the role of ICTs in supporting learning highlight the importance of teachers’ instructional practice and the learning context in the use of technologies and consequently their impact on instruction (Çelik, 2022 ; Schmid et al., 2014 ; Tamim et al., 2015 ). The review also provided useful insights regarding the various factors that affect the impact of digital technologies on education. These factors are interconnected and play a vital role in the transformation process. Specifically, these factors include a) digital competencies; b) teachers’ personal characteristics and professional development; c) school leadership and management; d) connectivity, infrastructure, and government support; e) administration and data management practices; f) students’ socio-economic background and family support and g) the socioeconomic context of the school and emergency situations. It is worth noting that we observed factors that affect the integration of ICTs in education but may also be affected by it. For example, the frequent use of ICTs and the use of laptops by students for instructional purposes positively affect the development of digital competencies (Zheng et al., 2016 ) and at the same time, the digital competencies affect the use of ICTs (Fu, 2013 ; Higgins et al., 2012 ). As a result, the impact of digital technologies should be explored more as an enabler of desirable and new practices and not merely as a catalyst that improves the output of the education process i.e. namely student attainment.

Conclusions

Digital technologies offer immense potential for fundamental improvement in schools. However, investment in ICT infrastructure and professional development to improve school education are yet to provide fruitful results. Digital transformation is a complex process that requires large-scale transformative changes that presuppose digital capacity and preparedness. To achieve such changes, all actors within the school’s ecosystem need to share a common vision regarding the integration of ICTs in education and work towards achieving this goal. Our literature review, which synthesized quantitative and qualitative data from a list of meta-analyses and review studies, provided useful insights into the impact of ICTs on different school stakeholders and showed that the impact of digital technologies touches upon many different aspects of school life, which are often overlooked when the focus is on student achievement as the final output of education. Furthermore, the concept of digital technologies is a concept in flux as technologies are not only different among them calling for different uses in the educational practice but they also change through time. Additionally, we opened a forum for discussion regarding the factors that affect a school’s digital capacity and transformation. We hope that our study will inform policy, practice, and research and result in a paradigm shift towards more holistic approaches in impact and assessment studies.

Study limitations and future directions

We presented a review of the study of digital technologies' impact on education and factors influencing schools’ digital capacity and transformation. The study results were based on a non-systematic literature review grounded on the acquisition of documentation in specific databases. Future studies should investigate more databases to corroborate and enhance our results. Moreover, search queries could be enhanced with key terms that could provide additional insights about the integration of ICTs in education, such as “policies and strategies for ICT integration in education”. Also, the study drew information from meta-analyses and literature reviews to acquire evidence about the effects of ICT integration in schools. Such evidence was mostly based on the general conclusions of the studies. It is worth mentioning that, we located individual studies which showed different, such as negative or neutral results. Thus, further insights are needed about the impact of ICTs on education and the factors influencing the impact. Furthermore, the nature of the studies included in meta-analyses and reviews is different as they are based on different research methodologies and data gathering processes. For instance, in a meta-analysis, the impact among the studies investigated is measured in a particular way, depending on policy or research targets (e.g., results from national examinations, pre-/post-tests). Meanwhile, in literature reviews, qualitative studies offer additional insights and detail based on self-reports and research opinions on several different aspects and stakeholders who could affect and be affected by ICT integration. As a result, it was challenging to draw causal relationships between so many interrelating variables.

Despite the challenges mentioned above, this study envisaged examining school units as ecosystems that consist of several actors by bringing together several variables from different research epistemologies to provide an understanding of the integration of ICTs. However, the use of other tools and methodologies and models for evaluation of the impact of digital technologies on education could give more detailed data and more accurate results. For instance, self-reflection tools, like SELFIE—developed on the DigCompOrg framework- (Kampylis et al., 2015 ; Bocconi & Lightfoot, 2021 ) can help capture a school’s digital capacity and better assess the impact of ICTs on education. Furthermore, the development of a theory of change could be a good approach for documenting the impact of digital technologies on education. Specifically, theories of change are models used for the evaluation of interventions and their impact; they are developed to describe how interventions will work and give the desired outcomes (Mayne, 2015 ). Theory of change as a methodological approach has also been used by researchers to develop models for evaluation in the field of education (e.g., Aromatario et al., 2019 ; Chapman & Sammons, 2013 ; De Silva et al., 2014 ).

We also propose that future studies aim at similar investigations by applying more holistic approaches for impact assessment that can provide in-depth data about the impact of digital technologies on education. For instance, future studies could focus on different research questions about the technologies that are used during the interventions or the way the implementation takes place (e.g., What methodologies are used for documenting impact? How are experimental studies implemented? How can teachers be taken into account and trained on the technology and its functions? What are the elements of an appropriate and successful implementation? How is the whole intervention designed? On which learning theories is the technology implementation based?).

Future research could also focus on assessing the impact of digital technologies on various other subjects since there is a scarcity of research related to particular subjects, such as geography, history, arts, music, and design and technology. More research should also be done about the impact of ICTs on skills, emotions, and attitudes, and on equality, inclusion, social interaction, and special needs education. There is also a need for more research about the impact of ICTs on administration, management, digitalization, and home-school relationships. Additionally, although new forms of teaching and learning with the use of ICTs (e.g., blended, hybrid, and online learning) have initiated several investigations in mainstream classrooms, only a few studies have measured their impact on students’ learning. Additionally, our review did not document any study about the impact of flipped classrooms on K-12 education. Regarding teaching and learning approaches, it is worth noting that studies referred to STEM or STEAM did not investigate the impact of STEM/STEAM as an interdisciplinary approach to learning but only investigated the impact of ICTs on learning in each domain as a separate subject (science, technology, engineering, arts, mathematics). Hence, we propose future research to also investigate the impact of the STEM/STEAM approach on education. The impact of emerging technologies on education, such as AR, VR, robotics, and AI has also been investigated recently, but more work needs to be done.

Finally, we propose that future studies could focus on the way in which specific factors, e.g., infrastructure and government support, school leadership and management, students’ and teachers’ digital competencies, approaches teachers utilize in the teaching and learning (e.g., blended, online and hybrid learning, flipped classrooms, STEM/STEAM approach, project-based learning, inquiry-based learning), affect the impact of digital technologies on education. We hope that future studies will give detailed insights into the concept of schools’ digital transformation through further investigation of impacts and factors which influence digital capacity and transformation based on the results and the recommendations of the present study.

Acknowledgements

This project has received funding under Grant Agreement No Ref Ares (2021) 339036 7483039 as well as funding from the European Union’s Horizon 2020 Research and Innovation Program under Grant Agreement No 739578 and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy. The UVa co-authors would like also to acknowledge funding from the European Regional Development Fund and the National Research Agency of the Spanish Ministry of Science and Innovation, under project grant PID2020-112584RB-C32.

Data availability statement

Declarations.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Archer K, Savage R, Sanghera-Sidhu S, Wood E, Gottardo A, Chen V. Examining the effectiveness of technology use in classrooms: A tertiary meta-analysis. Computers & Education. 2014; 78 :140–149. doi: 10.1016/j.compedu.2014.06.001. [ CrossRef ] [ Google Scholar ]
• Aromatario O, Van Hoye A, Vuillemin A, Foucaut AM, Pommier J, Cambon L. Using theory of change to develop an intervention theory for designing and evaluating behavior change SDApps for healthy eating and physical exercise: The OCAPREV theory. BMC Public Health. 2019; 19 (1):1–12. doi: 10.1186/s12889-019-7828-4. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Arztmann, M., Hornstra, L., Jeuring, J., & Kester, L. (2022). Effects of games in STEM education: A meta-analysis on the moderating role of student background characteristics. Studies in Science Education , 1-37. 10.1080/03057267.2022.2057732
• Bado N. Game-based learning pedagogy: A review of the literature. Interactive Learning Environments. 2022; 30 (5):936–948. doi: 10.1080/10494820.2019.1683587. [ CrossRef ] [ Google Scholar ]
• Balanskat, A. (2009). Study of the impact of technology in primary schools – Synthesis Report. Empirica and European Schoolnet. Retrieved 30 June 2022 from: https://erte.dge.mec.pt/sites/default/files/Recursos/Estudos/synthesis_report_steps_en.pdf
• Balanskat, A. (2006). The ICT Impact Report: A review of studies of ICT impact on schools in Europe, European Schoolnet. Retrieved 30 June 2022 from:  https://en.unesco.org/icted/content/ict-impact-report-review-studies-ict-impact-schools-europe
• Balanskat, A., Blamire, R., & Kefala, S. (2006). The ICT impact report.  European Schoolnet . Retrieved from: http://colccti.colfinder.org/sites/default/files/ict_impact_report_0.pdf
• Balyer, A., & Öz, Ö. (2018). Academicians’ views on digital transformation in education. International Online Journal of Education and Teaching (IOJET), 5 (4), 809–830. Retrieved 30 June 2022 from  http://iojet.org/index.php/IOJET/article/view/441/295
• Baragash RS, Al-Samarraie H, Moody L, Zaqout F. Augmented reality and functional skills acquisition among individuals with special needs: A meta-analysis of group design studies. Journal of Special Education Technology. 2022; 37 (1):74–81. doi: 10.1177/0162643420910413. [ CrossRef ] [ Google Scholar ]
• Bates, A. W. (2015). Teaching in a digital age: Guidelines for designing teaching and learning . Open Educational Resources Collection . 6. Retrieved 30 June 2022 from: https://irl.umsl.edu/oer/6
• Bingimlas KA. Barriers to the successful integration of ICT in teaching and learning environments: A review of the literature. Eurasia Journal of Mathematics, Science and Technology Education. 2009; 5 (3):235–245. doi: 10.12973/ejmste/75275. [ CrossRef ] [ Google Scholar ]
• Blaskó Z, Costa PD, Schnepf SV. Learning losses and educational inequalities in Europe: Mapping the potential consequences of the COVID-19 crisis. Journal of European Social Policy. 2022; 32 (4):361–375. doi: 10.1177/09589287221091687. [ CrossRef ] [ Google Scholar ]
• Bocconi S, Lightfoot M. Scaling up and integrating the selfie tool for schools' digital capacity in education and training systems: Methodology and lessons learnt. European Training Foundation. 2021 doi: 10.2816/907029,JRC123936. [ CrossRef ] [ Google Scholar ]
• Brooks, D. C., & McCormack, M. (2020). Driving Digital Transformation in Higher Education . Retrieved 30 June 2022 from: https://library.educause.edu/-/media/files/library/2020/6/dx2020.pdf?la=en&hash=28FB8C377B59AFB1855C225BBA8E3CFBB0A271DA
• Cachia, R., Chaudron, S., Di Gioia, R., Velicu, A., & Vuorikari, R. (2021). Emergency remote schooling during COVID-19, a closer look at European families. Retrieved 30 June 2022 from  https://publications.jrc.ec.europa.eu/repository/handle/JRC125787
• Çelik B. The effects of computer simulations on students’ science process skills: Literature review. Canadian Journal of Educational and Social Studies. 2022; 2 (1):16–28. doi: 10.53103/cjess.v2i1.17. [ CrossRef ] [ Google Scholar ]
• Chapman, C., & Sammons, P. (2013). School Self-Evaluation for School Improvement: What Works and Why? . CfBT Education Trust. 60 Queens Road, Reading, RG1 4BS, England.
• Chauhan S. A meta-analysis of the impact of technology on learning effectiveness of elementary students. Computers & Education. 2017; 105 :14–30. doi: 10.1016/j.compedu.2016.11.005. [ CrossRef ] [ Google Scholar ]
• Chen, Q., Chan, K. L., Guo, S., Chen, M., Lo, C. K. M., & Ip, P. (2022a). Effectiveness of digital health interventions in reducing bullying and cyberbullying: a meta-analysis. Trauma, Violence, & Abuse , 15248380221082090. 10.1177/15248380221082090 [ PubMed ]
• Chen B, Wang Y, Wang L. The effects of virtual reality-assisted language learning: A meta-analysis. Sustainability. 2022; 14 (6):3147. doi: 10.3390/su14063147. [ CrossRef ] [ Google Scholar ]
• Cheok ML, Wong SL. Predictors of e-learning satisfaction in teaching and learning for school teachers: A literature review. International Journal of Instruction. 2015; 8 (1):75–90. doi: 10.12973/iji.2015.816a. [ CrossRef ] [ Google Scholar ]
• Cheung, A. C., & Slavin, R. E. (2011). The Effectiveness of Education Technology for Enhancing Reading Achievement: A Meta-Analysis. Center for Research and reform in Education .
• Coban, M., Bolat, Y. I., & Goksu, I. (2022). The potential of immersive virtual reality to enhance learning: A meta-analysis. Educational Research Review , 100452. 10.1016/j.edurev.2022.100452
• Condie, R., & Munro, R. K. (2007). The impact of ICT in schools-a landscape review. Retrieved 30 June 2022 from: https://oei.org.ar/ibertic/evaluacion/sites/default/files/biblioteca/33_impact_ict_in_schools.pdf
• Conrads, J., Rasmussen, M., Winters, N., Geniet, A., Langer, L., (2017). Digital Education Policies in Europe and Beyond: Key Design Principles for More Effective Policies. Redecker, C., P. Kampylis, M. Bacigalupo, Y. Punie (ed.), EUR 29000 EN, Publications Office of the European Union, Luxembourg, 10.2760/462941
• Costa P, Castaño-Muñoz J, Kampylis P. Capturing schools’ digital capacity: Psychometric analyses of the SELFIE self-reflection tool. Computers & Education. 2021; 162 :104080. doi: 10.1016/j.compedu.2020.104080. [ CrossRef ] [ Google Scholar ]
• Cussó-Calabuig R, Farran XC, Bosch-Capblanch X. Effects of intensive use of computers in secondary school on gender differences in attitudes towards ICT: A systematic review. Education and Information Technologies. 2018; 23 (5):2111–2139. doi: 10.1007/s10639-018-9706-6. [ CrossRef ] [ Google Scholar ]
• Daniel SJ. Education and the COVID-19 pandemic. Prospects. 2020; 49 (1):91–96. doi: 10.1007/s11125-020-09464-3. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Delcker J, Ifenthaler D. Teachers’ perspective on school development at German vocational schools during the Covid-19 pandemic. Technology, Pedagogy and Education. 2021; 30 (1):125–139. doi: 10.1080/1475939X.2020.1857826. [ CrossRef ] [ Google Scholar ]
• Delgado, A., Wardlow, L., O’Malley, K., & McKnight, K. (2015). Educational technology: A review of the integration, resources, and effectiveness of technology in K-12 classrooms. Journal of Information Technology Education Research , 14, 397. Retrieved 30 June 2022 from  http://www.jite.org/documents/Vol14/JITEv14ResearchP397-416Delgado1829.pdf
• De Silva MJ, Breuer E, Lee L, Asher L, Chowdhary N, Lund C, Patel V. Theory of change: A theory-driven approach to enhance the Medical Research Council's framework for complex interventions. Trials. 2014; 15 (1):1–13. doi: 10.1186/1745-6215-15-267. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Di Pietro G, Biagi F, Costa P, Karpiński Z, Mazza J. The likely impact of COVID-19 on education: Reflections based on the existing literature and recent international datasets. Publications Office of the European Union; 2020. [ Google Scholar ]
• Elkordy A, Lovinelli J. Competencies, Culture, and Change: A Model for Digital Transformation in K12 Educational Contexts. In: Ifenthaler D, Hofhues S, Egloffstein M, Helbig C, editors. Digital Transformation of Learning Organizations. Springer; 2020. pp. 203–219. [ Google Scholar ]
• Eng TS. The impact of ICT on learning: A review of research. International Education Journal. 2005; 6 (5):635–650. [ Google Scholar ]
• European Commission. (2020). Digital Education Action Plan 2021 – 2027. Resetting education and training for the digital age. Retrieved 30 June 2022 from  https://ec.europa.eu/education/sites/default/files/document-library-docs/deap-communication-sept2020_en.pdf
• European Commission. (2019). 2 nd survey of schools: ICT in education. Objective 1: Benchmark progress in ICT in schools . Retrieved 30 June 2022 from: https://data.europa.eu/euodp/data/storage/f/2019-03-19T084831/FinalreportObjective1-BenchmarkprogressinICTinschools.pdf
• Eurydice. (2019). Digital Education at School in Europe , Luxembourg: Publications Office of the European Union. Retrieved 30 June 2022 from: https://eacea.ec.europa.eu/national-policies/eurydice/content/digital-education-school-europe_en
• Escueta, M., Quan, V., Nickow, A. J., & Oreopoulos, P. (2017). Education technology: An evidence-based review. Retrieved 30 June 2022 from  https://ssrn.com/abstract=3031695
• Fadda D, Pellegrini M, Vivanet G, Zandonella Callegher C. Effects of digital games on student motivation in mathematics: A meta-analysis in K-12. Journal of Computer Assisted Learning. 2022; 38 (1):304–325. doi: 10.1111/jcal.12618. [ CrossRef ] [ Google Scholar ]
• Fernández-Gutiérrez M, Gimenez G, Calero J. Is the use of ICT in education leading to higher student outcomes? Analysis from the Spanish Autonomous Communities. Computers & Education. 2020; 157 :103969. doi: 10.1016/j.compedu.2020.103969. [ CrossRef ] [ Google Scholar ]
• Ferrari, A., Cachia, R., & Punie, Y. (2011). Educational change through technology: A challenge for obligatory schooling in Europe. Lecture Notes in Computer Science , 6964 , 97–110. Retrieved 30 June 2022  https://link.springer.com/content/pdf/10.1007/978-3-642-23985-4.pdf
• Fielding, K., & Murcia, K. (2022). Research linking digital technologies to young children’s creativity: An interpretive framework and systematic review. Issues in Educational Research , 32 (1), 105–125. Retrieved 30 June 2022 from  http://www.iier.org.au/iier32/fielding-abs.html
• Friedel, H., Bos, B., Lee, K., & Smith, S. (2013). The impact of mobile handheld digital devices on student learning: A literature review with meta-analysis. In Society for Information Technology & Teacher Education International Conference (pp. 3708–3717). Association for the Advancement of Computing in Education (AACE).
• Fu JS. ICT in education: A critical literature review and its implications. International Journal of Education and Development Using Information and Communication Technology (IJEDICT) 2013; 9 (1):112–125. [ Google Scholar ]
• Gaol FL, Prasolova-Førland E. Special section editorial: The frontiers of augmented and mixed reality in all levels of education. Education and Information Technologies. 2022; 27 (1):611–623. doi: 10.1007/s10639-021-10746-2. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Garzón J, Acevedo J. Meta-analysis of the impact of Augmented Reality on students’ learning gains. Educational Research Review. 2019; 27 :244–260. doi: 10.1016/j.edurev.2019.04.001. [ CrossRef ] [ Google Scholar ]
• Garzón, J., Baldiris, S., Gutiérrez, J., & Pavón, J. (2020). How do pedagogical approaches affect the impact of augmented reality on education? A meta-analysis and research synthesis. Educational Research Review , 100334. 10.1016/j.edurev.2020.100334
• Grgurović M, Chapelle CA, Shelley MC. A meta-analysis of effectiveness studies on computer technology-supported language learning. ReCALL. 2013; 25 (2):165–198. doi: 10.1017/S0958344013000013. [ CrossRef ] [ Google Scholar ]
• Haßler B, Major L, Hennessy S. Tablet use in schools: A critical review of the evidence for learning outcomes. Journal of Computer Assisted Learning. 2016; 32 (2):139–156. doi: 10.1111/jcal.12123. [ CrossRef ] [ Google Scholar ]
• Haleem A, Javaid M, Qadri MA, Suman R. Understanding the role of digital technologies in education: A review. Sustainable Operations and Computers. 2022; 3 :275–285. doi: 10.1016/j.susoc.2022.05.004. [ CrossRef ] [ Google Scholar ]
• Hardman J. Towards a pedagogical model of teaching with ICTs for mathematics attainment in primary school: A review of studies 2008–2018. Heliyon. 2019; 5 (5):e01726. doi: 10.1016/j.heliyon.2019.e01726. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hattie J, Rogers HJ, Swaminathan H. The role of meta-analysis in educational research. In: Reid AD, Hart P, Peters MA, editors. A companion to research in education. Springer; 2014. pp. 197–207. [ Google Scholar ]
• Hattie J. Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge. 2008 doi: 10.4324/9780203887332. [ CrossRef ] [ Google Scholar ]
• Higgins S, Xiao Z, Katsipataki M. The impact of digital technology on learning: A summary for the education endowment foundation. Education Endowment Foundation and Durham University; 2012. [ Google Scholar ]
• Higgins, K., Huscroft-D’Angelo, J., & Crawford, L. (2019). Effects of technology in mathematics on achievement, motivation, and attitude: A meta-analysis. Journal of Educational Computing Research , 57(2), 283-319.
• Hillmayr D, Ziernwald L, Reinhold F, Hofer SI, Reiss KM. The potential of digital tools to enhance mathematics and science learning in secondary schools: A context-specific meta-analysis. Computers & Education. 2020; 153 (1038):97. doi: 10.1016/j.compedu.2020.103897. [ CrossRef ] [ Google Scholar ]
• Istenic Starcic A, Bagon S. ICT-supported learning for inclusion of people with special needs: Review of seven educational technology journals, 1970–2011. British Journal of Educational Technology. 2014; 45 (2):202–230. doi: 10.1111/bjet.12086. [ CrossRef ] [ Google Scholar ]
• Jewitt C, Clark W, Hadjithoma-Garstka C. The use of learning platforms to organise learning in English primary and secondary schools. Learning, Media and Technology. 2011; 36 (4):335–348. doi: 10.1080/17439884.2011.621955. [ CrossRef ] [ Google Scholar ]
• JISC. (2020). What is digital transformation?.  Retrieved 30 June 2022 from: https://www.jisc.ac.uk/guides/digital-strategy-framework-for-university-leaders/what-is-digital-transformation
• Kalati, A. T., & Kim, M. S. (2022). What is the effect of touchscreen technology on young children’s learning?: A systematic review. Education and Information Technologies , 1-19. 10.1007/s10639-021-10816-5
• Kalemkuş, J., & Kalemkuş, F. (2022). Effect of the use of augmented reality applications on academic achievement of student in science education: Meta-analysis review. Interactive Learning Environments , 1-18. 10.1080/10494820.2022.2027458
• Kao C-W. The effects of digital game-based learning task in English as a foreign language contexts: A meta-analysis. Education Journal. 2014; 42 (2):113–141. [ Google Scholar ]
• Kampylis P, Punie Y, Devine J. Promoting effective digital-age learning - a European framework for digitally competent educational organisations. JRC Technical Reports. 2015 doi: 10.2791/54070. [ CrossRef ] [ Google Scholar ]
• Kazu IY, Yalçin CK. Investigation of the effectiveness of hybrid learning on academic achievement: A meta-analysis study. International Journal of Progressive Education. 2022; 18 (1):249–265. doi: 10.29329/ijpe.2022.426.14. [ CrossRef ] [ Google Scholar ]
• Koh C. A qualitative meta-analysis on the use of serious games to support learners with intellectual and developmental disabilities: What we know, what we need to know and what we can do. International Journal of Disability, Development and Education. 2022; 69 (3):919–950. doi: 10.1080/1034912X.2020.1746245. [ CrossRef ] [ Google Scholar ]
• König J, Jäger-Biela DJ, Glutsch N. Adapting to online teaching during COVID-19 school closure: Teacher education and teacher competence effects among early career teachers in Germany. European Journal of Teacher Education. 2020; 43 (4):608–622. doi: 10.1080/02619768.2020.1809650. [ CrossRef ] [ Google Scholar ]
• Lawrence JE, Tar UA. Factors that influence teachers’ adoption and integration of ICT in teaching/learning process. Educational Media International. 2018; 55 (1):79–105. doi: 10.1080/09523987.2018.1439712. [ CrossRef ] [ Google Scholar ]
• Lee, S., Kuo, L. J., Xu, Z., & Hu, X. (2020). The effects of technology-integrated classroom instruction on K-12 English language learners’ literacy development: A meta-analysis. Computer Assisted Language Learning , 1-32. 10.1080/09588221.2020.1774612
• Lei, H., Chiu, M. M., Wang, D., Wang, C., & Xie, T. (2022a). Effects of game-based learning on students’ achievement in science: a meta-analysis. Journal of Educational Computing Research . 10.1177/07356331211064543
• Lei H, Wang C, Chiu MM, Chen S. Do educational games affect students' achievement emotions? Evidence from a meta-analysis. Journal of Computer Assisted Learning. 2022; 38 (4):946–959. doi: 10.1111/jcal.12664. [ CrossRef ] [ Google Scholar ]
• Liao YKC, Chang HW, Chen YW. Effects of computer application on elementary school student's achievement: A meta-analysis of students in Taiwan. Computers in the Schools. 2007; 24 (3–4):43–64. doi: 10.1300/J025v24n03_04. [ CrossRef ] [ Google Scholar ]
• Li Q, Ma X. A meta-analysis of the effects of computer technology on school students’ mathematics learning. Educational Psychology Review. 2010; 22 (3):215–243. doi: 10.1007/s10648-010-9125-8. [ CrossRef ] [ Google Scholar ]
• Liu, M., Pang, W., Guo, J., & Zhang, Y. (2022). A meta-analysis of the effect of multimedia technology on creative performance. Education and Information Technologies , 1-28. 10.1007/s10639-022-10981-1
• Lu Z, Chiu MM, Cui Y, Mao W, Lei H. Effects of game-based learning on students’ computational thinking: A meta-analysis. Journal of Educational Computing Research. 2022 doi: 10.1177/07356331221100740. [ CrossRef ] [ Google Scholar ]
• Martinez L, Gimenes M, Lambert E. Entertainment video games for academic learning: A systematic review. Journal of Educational Computing Research. 2022 doi: 10.1177/07356331211053848. [ CrossRef ] [ Google Scholar ]
• Mayne J. Useful theory of change models. Canadian Journal of Program Evaluation. 2015; 30 (2):119–142. doi: 10.3138/cjpe.230. [ CrossRef ] [ Google Scholar ]
• Moran J, Ferdig RE, Pearson PD, Wardrop J, Blomeyer RL., Jr Technology and reading performance in the middle-school grades: A meta-analysis with recommendations for policy and practice. Journal of Literacy Research. 2008; 40 (1):6–58. doi: 10.1080/10862960802070483. [ CrossRef ] [ Google Scholar ]
• OECD. (2015). Students, Computers and Learning: Making the Connection . PISA, OECD Publishing, Paris. Retrieved from: 10.1787/9789264239555-en
• OECD. (2021). OECD Digital Education Outlook 2021: Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots. Retrieved from: https://www.oecd-ilibrary.org/education/oecd-digital-education-outlook-2021_589b283f-en
• Pan Y, Ke F, Xu X. A systematic review of the role of learning games in fostering mathematics education in K-12 settings. Educational Research Review. 2022; 36 :100448. doi: 10.1016/j.edurev.2022.100448. [ CrossRef ] [ Google Scholar ]
• Pettersson F. Understanding digitalization and educational change in school by means of activity theory and the levels of learning concept. Education and Information Technologies. 2021; 26 (1):187–204. doi: 10.1007/s10639-020-10239-8. [ CrossRef ] [ Google Scholar ]
• Pihir, I., Tomičić-Pupek, K., & Furjan, M. T. (2018). Digital transformation insights and trends. In Central European Conference on Information and Intelligent Systems (pp. 141–149). Faculty of Organization and Informatics Varazdin. Retrieved 30 June 2022 from https://www.proquest.com/conference-papers-proceedings/digital-transformation-insights-trends/docview/2125639934/se-2
• Punie, Y., Zinnbauer, D., & Cabrera, M. (2006). A review of the impact of ICT on learning. Working Paper prepared for DG EAC. Retrieved 30 June 2022 from: http://www.eurosfaire.prd.fr/7pc/doc/1224678677_jrc47246n.pdf
• Quah CY, Ng KH. A systematic literature review on digital storytelling authoring tool in education: January 2010 to January 2020. International Journal of Human-Computer Interaction. 2022; 38 (9):851–867. doi: 10.1080/10447318.2021.1972608. [ CrossRef ] [ Google Scholar ]
• Ran H, Kim NJ, Secada WG. A meta-analysis on the effects of technology's functions and roles on students' mathematics achievement in K-12 classrooms. Journal of computer assisted learning. 2022; 38 (1):258–284. doi: 10.1111/jcal.12611. [ CrossRef ] [ Google Scholar ]
• Ređep, N. B. (2021). Comparative overview of the digital preparedness of education systems in selected CEE countries. Center for Policy Studies. CEU Democracy Institute .
• Rott, B., & Marouane, C. (2018). Digitalization in schools–organization, collaboration and communication. In Digital Marketplaces Unleashed (pp. 113–124). Springer, Berlin, Heidelberg.
• Savva M, Higgins S, Beckmann N. Meta-analysis examining the effects of electronic storybooks on language and literacy outcomes for children in grades Pre-K to grade 2. Journal of Computer Assisted Learning. 2022; 38 (2):526–564. doi: 10.1111/jcal.12623. [ CrossRef ] [ Google Scholar ]
• Schmid RF, Bernard RM, Borokhovski E, Tamim RM, Abrami PC, Surkes MA, Wade CA, Woods J. The effects of technology use in postsecondary education: A meta-analysis of classroom applications. Computers & Education. 2014; 72 :271–291. doi: 10.1016/j.compedu.2013.11.002. [ CrossRef ] [ Google Scholar ]
• Schuele CM, Justice LM. The importance of effect sizes in the interpretation of research: Primer on research: Part 3. The ASHA Leader. 2006; 11 (10):14–27. doi: 10.1044/leader.FTR4.11102006.14. [ CrossRef ] [ Google Scholar ]
• Schwabe, A., Lind, F., Kosch, L., & Boomgaarden, H. G. (2022). No negative effects of reading on screen on comprehension of narrative texts compared to print: A meta-analysis. Media Psychology , 1-18. 10.1080/15213269.2022.2070216
• Sellar S. Data infrastructure: a review of expanding accountability systems and large-scale assessments in education. Discourse: Studies in the Cultural Politics of Education. 2015; 36 (5):765–777. doi: 10.1080/01596306.2014.931117. [ CrossRef ] [ Google Scholar ]
• Stock WA. Systematic coding for research synthesis. In: Cooper H, Hedges LV, editors. The handbook of research synthesis, 236. Russel Sage; 1994. pp. 125–138. [ Google Scholar ]
• Su, J., Zhong, Y., & Ng, D. T. K. (2022). A meta-review of literature on educational approaches for teaching AI at the K-12 levels in the Asia-Pacific region. Computers and Education: Artificial Intelligence , 100065. 10.1016/j.caeai.2022.100065
• Su J, Yang W. Artificial intelligence in early childhood education: A scoping review. Computers and Education: Artificial Intelligence. 2022; 3 :100049. doi: 10.1016/j.caeai.2022.100049. [ CrossRef ] [ Google Scholar ]
• Sung YT, Chang KE, Liu TC. The effects of integrating mobile devices with teaching and learning on students' learning performance: A meta-analysis and research synthesis. Computers & Education. 2016; 94 :252–275. doi: 10.1016/j.compedu.2015.11.008. [ CrossRef ] [ Google Scholar ]
• Talan T, Doğan Y, Batdı V. Efficiency of digital and non-digital educational games: A comparative meta-analysis and a meta-thematic analysis. Journal of Research on Technology in Education. 2020; 52 (4):474–514. doi: 10.1080/15391523.2020.1743798. [ CrossRef ] [ Google Scholar ]
• Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational research, 81 (1), 4–28. Retrieved 30 June 2022 from 10.3102/0034654310393361
• Tamim, R. M., Borokhovski, E., Pickup, D., Bernard, R. M., & El Saadi, L. (2015). Tablets for teaching and learning: A systematic review and meta-analysis. Commonwealth of Learning. Retrieved from: http://oasis.col.org/bitstream/handle/11599/1012/2015_Tamim-et-al_Tablets-for-Teaching-and-Learning.pdf
• Tang C, Mao S, Xing Z, Naumann S. Improving student creativity through digital technology products: A literature review. Thinking Skills and Creativity. 2022; 44 :101032. doi: 10.1016/j.tsc.2022.101032. [ CrossRef ] [ Google Scholar ]
• Tolani-Brown, N., McCormac, M., & Zimmermann, R. (2011). An analysis of the research and impact of ICT in education in developing country contexts. In ICTs and sustainable solutions for the digital divide: Theory and perspectives (pp. 218–242). IGI Global.
• Trucano, M. (2005). Knowledge Maps: ICTs in Education. Washington, DC: info Dev / World Bank. Retrieved 30 June 2022 from  https://files.eric.ed.gov/fulltext/ED496513.pdf
• Ulum H. The effects of online education on academic success: A meta-analysis study. Education and Information Technologies. 2022; 27 (1):429–450. doi: 10.1007/s10639-021-10740-8. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Underwood, J. D. (2009). The impact of digital technology: A review of the evidence of the impact of digital technologies on formal education. Retrieved 30 June 2022 from: http://dera.ioe.ac.uk/id/eprint/10491
• Verschaffel, L., Depaepe, F., & Mevarech, Z. (2019). Learning Mathematics in metacognitively oriented ICT-Based learning environments: A systematic review of the literature. Education Research International , 2019 . 10.1155/2019/3402035
• Villena-Taranilla R, Tirado-Olivares S, Cózar-Gutiérrez R, González-Calero JA. Effects of virtual reality on learning outcomes in K-6 education: A meta-analysis. Educational Research Review. 2022; 35 :100434. doi: 10.1016/j.edurev.2022.100434. [ CrossRef ] [ Google Scholar ]
• Voogt J, Knezek G, Cox M, Knezek D, ten Brummelhuis A. Under which conditions does ICT have a positive effect on teaching and learning? A call to action. Journal of Computer Assisted Learning. 2013; 29 (1):4–14. doi: 10.1111/j.1365-2729.2011.00453.x. [ CrossRef ] [ Google Scholar ]
• Vuorikari, R., Punie, Y., & Cabrera, M. (2020). Emerging technologies and the teaching profession: Ethical and pedagogical considerations based on near-future scenarios  (No. JRC120183). Joint Research Centre. Retrieved 30 June 2022 from: https://publications.jrc.ec.europa.eu/repository/handle/JRC120183
• Wang LH, Chen B, Hwang GJ, Guan JQ, Wang YQ. Effects of digital game-based STEM education on students’ learning achievement: A meta-analysis. International Journal of STEM Education. 2022; 9 (1):1–13. doi: 10.1186/s40594-022-00344-0. [ CrossRef ] [ Google Scholar ]
• Wen X, Walters SM. The impact of technology on students’ writing performances in elementary classrooms: A meta-analysis. Computers and Education Open. 2022; 3 :100082. doi: 10.1016/j.caeo.2022.100082. [ CrossRef ] [ Google Scholar ]
• Zheng B, Warschauer M, Lin CH, Chang C. Learning in one-to-one laptop environments: A meta-analysis and research synthesis. Review of Educational Research. 2016; 86 (4):1052–1084. doi: 10.3102/0034654316628645. [ CrossRef ] [ Google Scholar ]

ScholarWorks@GVSU

• < Previous

Home > Graduate Research and Creative Practice > Culminating Experience Projects > 407

Culminating Experience Projects

Culturally responsive education: creating a safe and inclusive environment for lgbtq+ students.

Kassidi M. Adams , Grand Valley State University Follow

Date Approved

Graduate degree type, degree name.

Education-Literacy Studies: Reading (M.Ed.)

Degree Program

College of Education

First Advisor

Elizabeth Stolle Ph.D

Second Advisor

Mary Bair Ph. D

Academic Year

This paper explores research that discusses the impact of a culturally responsive education and lack-of for LGBTQ+ students and recommends a solution to ensure all students are safe and represented in schools. The research cited varies in their definitions and uses of the term LGBTQ+. This paper will use the term LGBTQ+, an abbreviation for lesbian, gay, bisexual, transgender, queer, and more communities; however, the abbreviation used in the referenced research may be shortened to LG, LGB, LGBT, or LGBTQ to reflect only specific communities included in LGBTTTQQIAAP (VandenBos, 2015). This paper will also focus on inclusion, which refers to increasing learning and participation for all students. Inclusion involves valuing all students and staff equally, increasing the participation of students, and removing barriers to learning and participation (Booth & Ainscow, 2002). Finally, this project will discuss the importance of anti-bias education through the lens of culturally relevant teaching. Anti-bias education refers to supporting children to develop: a confident sense of identity without needing to feel superior to others; an ease with human diversity; a sense of fairness and justice; the skills of empowerment; and the ability to stand up for themselves and for others (Derman-Sparks and Edwards, 2019). When using the terms anti-bias or LGBTQ+ inclusive, this paper does not wish to de-center race or ethnicity but rather focus on the role gender and sexuality play with the understanding that social identities are intersectional.

ScholarWorks Citation

Adams, Kassidi M., "Culturally Responsive Education: Creating a Safe and Inclusive Environment for LGBTQ+ Students" (2023). Culminating Experience Projects . 407. https://scholarworks.gvsu.edu/gradprojects/407

Since December 15, 2023

Included in

Elementary Education and Teaching Commons

Advanced Search

• Notify me via email or RSS
• Collections
• University Archives
• Open Textbooks
• Open Educational Resources
• Graduate Research and Creative Practice
• Selected Works Galleries

Author Information

• Submission Guidelines
• Submit Research
• Graduate Student Resources

Home | About | FAQ | Contact | My Account | Accessibility Statement

Privacy Copyright

Diving Deeper into Postsecondary Value, IHEP Research Series Explores Equity and Economic Outcomes

Washington, DC (May 29, 2024) –   Higher education has long been recognized as a key driver of economic opportunity, but new research, spearheaded by the Institute for Higher Education Policy (IHEP), is diving deeper into questions of postsecondary value and equity. The “Elevating Equitable Value: Investigating Economic Outcomes of Postsecondary Education” series, informed by data from the Equitable Value Explorer tool along with state-level data, surveys, and additional sources, is exploring pressing questions about postsecondary value. Research released today by Trellis Strategies, the American Association of State Colleges and Universities (AASCU), Wayne State University, and the Research Institute at Dallas College completes the seven-paper series.  

“Earning a college degree can change the trajectory of students’ lives, their families, and their communities, for generations to come, but the benefits are not evenly distributed,” said IHEP President Mamie Voight . “By unpacking the nuances of value delivery across different contexts, this research strengthens the evidence-base showing that college is worth the investment. It also can inform policymakers and institutions about targeted strategies to improve the returns on postsecondary education for all students.”  

Field-based researchers across the country built on the work of the Postsecondary Value Commission by exploring critical questions on post-college outcomes in their own specific context:  

How Can Policy and Practice Shape Equitable Value?

Several papers in the series explored how institutional policy and practice can ensure all students benefit from a college degree. Research by Trellis Strategies underscores the connection between student financial well-being during college and future economic returns. A ten-percentage-point decrease in the number of students experiencing financial instability during college was correlated with higher economic returns to students, especially at four-year public schools. The authors highlight the importance of programs that expand access to emergency aid, promote financial literacy, and enhance transparency around college costs and financial aid options.  

A study by the American Association of State Colleges and Universities (AASCU) found that all 33 institutions participating in their Student Success Equity Intensive program had earnings that exceeded the Postsecondary Value Commission’s minimum economic return threshold, but equity gaps persist. Institutions serving a larger proportion of Black, Latinx, and Indigenous students or Pell Grant recipients saw a smaller economic return – the amount by which earnings exceed those of a typical high school graduate, plus the cost of their education  – demonstrating the need for continued efforts to promote student success and the financial value of college degrees.  

Another key finding shows a strong link between faculty composition and student outcomes. Research by Frederick Tucker of the City University of New York reveals that institutions with more tenured or tenure-track faculty, alongside a smaller proportion of full-time adjuncts, see stronger economic returns for students. This is particularly true at Minority-Serving Institutions (MSIs) and colleges serving a large share of Pell Grant recipients.  

How Does Postsecondary Value Differ By and Within States?

The research in this series adds to our growing understanding of how certain states are delivering value. For example, a project by Wayne State University found that while postsecondary education generally leads to higher earnings in Michigan, disparities exist. Public, four-year universities in the state provided the highest economic return, exceeding the minimum threshold by over $22,000, while most other types of institutions in the state offered a smaller but still positive return. Michigan ranked high nationally in terms of the amount that students’ earnings typically exceed the minimum economic return threshold,  despite having a moderate ranking in median post-college earnings.  

The Research Institute at Dallas College’s analysis paired data from the Equitable Value Explorer with state longitudinal data to measure the economic return at more than 500 Hispanic-Serving Institutions (HSIs) nationwide, in addition to outcomes for all Hispanic students in Texas. Overall, economic returns were positive for Hispanic students, but disparities persisted even within Hispanic communities, particularly for immigrants, those from low-income backgrounds and women.   

What is the Role of Geography in the Postsecondary Value Equation?

Location plays a significant role in education options and economic returns. Two papers in the series examined the geographic dimension of value. The American Institutes for Research’s study found that community college students generally earn more after college than those with only a high school diploma in their region. However, community colleges serving a higher percentage of underrepresented students tend to deliver a lower economic value, underscoring the need for additional support to ensure these institutions can effectively serve their students.  

Boston College researchers’ analysis of rural-serving institutions (RSIs) found that while median post-college earnings may be slightly lower compared to urban and suburban institutions, RSIs are typically more affordable. The majority of RSIs still provide a positive economic return for students after factoring in these relatively low education costs, highlighting their vital role in expanding access to higher education in underserved areas.  

IHEP is expanding and strengthening the community of researchers, practitioners and advocates exploring postsecondary value through an equity lens, by providing the tools necessary for researchers, associations, and institutions to tackle these critical questions. For more information about the Equitable Value Explorer, please visit: https://equity.postsecondaryvalue.org/     

Supporting the Whole Student Through Holistic Advising: Reflections on ED’s Raise The Bar Summit

Investing in Student Success: IHEP’s Federal Funding Priorities for FY25

Along with Stanford news and stories, show me:

• Student information
• Faculty/Staff information

We want to provide announcements, events, leadership messages and resources that are relevant to you. Your selection is stored in a browser cookie which you can remove at any time using “Clear all personalization” below.

Artificial intelligence is rapidly transforming education, in both worrisome and beneficial ways. On the positive side of the ledger, new research shows how AI can help improve the way instructors engage with their students, by way of a cutting-edge tool that provides feedback on their interactions in class.

The M-Powering Teachers tool provides feedback with examples of dialogue from the class to illustrate supportive conversational patterns. Click on image to enlarge. (Image credit: Courtesy Dora Demszky)

A new Stanford-led study , published May 8 in the peer-reviewed journal Educational Evaluation and Policy Analysis, found that an automated feedback tool improved instructors’ use of a practice known as uptake, where teachers acknowledge, reiterate, and build on students’ contributions. The findings also provided evidence that, among students, the tool improved their rate of completing assignments and their overall satisfaction with the course.

For instructors looking to improve their practice, the tool offers a low-cost complement to conventional classroom observation – one that doesn’t require an instructional coach or other expert to watch the teacher in action and compile a set of recommendations.

“We know from past research that timely, specific feedback can improve teaching, but it’s just not scalable or feasible for someone to sit in a teacher’s classroom and give feedback every time,” said Dora Demszky , an assistant professor at Stanford Graduate School of Education (GSE) and lead author of the study. “We wanted to see whether an automated tool could support teachers’ professional development in a scalable and cost-effective way, and this is the first study to show that it does.”

Promoting effective teaching practices

Recognizing that existing methods for providing personalized feedback require significant resources, Demszky and colleagues set out to create a low-cost alternative. They leveraged recent advances in natural language processing (NLP) – a branch of AI that helps computers read and interpret human language – to develop a tool that could analyze transcripts of a class session to identify conversational patterns and deliver consistent, automated feedback.

For this study, they focused on identifying teachers’ uptake of student contributions. “Uptake is key to making students feel heard, and as a practice it’s been linked to greater student achievement,” said Demszky. “But it’s also widely considered difficult for teachers to improve.”

Dora Demszky is an assistant professor at Stanford Graduate School of Education. (Image credit: Courtesy Stanford GSE)

The researchers trained the tool, called M-Powering Teachers (the M stands for machine, as in machine learning), to detect the extent to which a teacher’s response is specific to what a student has said, which would show that the teacher understood and built on the student’s idea. The tool can also provide feedback on teachers’ questioning practices, such as posing questions that elicited a significant response from students, and the ratio of teacher/student talk time.

The research team put the tool to work in the Spring 2021 session of Stanford’s Code in Place , a free online course now in its third year. In the five-week program, based on Stanford’s popular introductory computer science course, hundreds of volunteer instructors teach basic programming to learners worldwide, in small sections with a 1:10 teacher-student ratio.

Code in Place instructors come from all sorts of backgrounds, from undergrads who’ve recently taken the course themselves to professional computer programmers working in the industry. Enthusiastic as they are to introduce beginners to the world of coding, many instructors approach the opportunity with little or no prior teaching experience.

The volunteer instructors received basic training, clear lesson goals, and session outlines to prepare for their role, and many welcomed the chance to receive automated input on their sessions, said study co-author Chris Piech , an assistant professor of computer science education at Stanford and co-founder of Code in Place.

“We make such a big deal in education about the importance of timely feedback for students, but when do teachers get that kind of feedback?” he said. “Maybe the principal will come in and sit in on your class, which seems terrifying. It’s much more comfortable to engage with feedback that’s not coming from your principal, and you can get it not just after years of practice but from your first day on the job.”

Instructors received their feedback from the tool through an app within a few days after each class, so they could reflect on it before the next session. Presented in a colorful, easy-to-read format, the feedback used positive, nonjudgmental language and included specific examples of dialogue from their class to illustrate supportive conversational patterns.

The researchers found that, on average, instructors who reviewed their feedback subsequently increased their use of uptake and questioning, with the most significant changes taking place in the third week of the course. Student learning and satisfaction with the course also increased among those whose instructors received feedback, compared with the control group. Code in Place doesn’t administer an end-of-course exam, so the researchers used the completion rates of optional assignments and course surveys to measure student learning and satisfaction.

Testing in other settings

Subsequent research by Demszky with one of the study’s coauthors, Jing Liu, PhD ’18, studied the use of the tool among instructors who worked one-on-one with high school students in an online mentoring program called Polygence . The researchers, who will present their findings in July at the 2023 Learning at Scale conference, found that on average the tool improved mentors’ uptake of student contributions by 10%, reduced their talk time by 5%, and improved students’ experience with the program as well as their relative optimism about their academic future.

Demszky is currently conducting a study of the tool’s use for in-person, K-12 school classrooms, and she noted the challenge of generating the high-quality transcription she was able to obtain from a virtual setting. “The audio quality from the classroom is not great, and separating voices is not easy,” she said. “Natural language processing can do so much once you have the transcripts – but you need good transcripts.”

She stressed that the tool was not designed for surveillance or evaluation purposes, but to support teachers’ professional development by giving them an opportunity to reflect on their practices. She likened it to a fitness tracker, providing information for its users’ own benefit.

The tool also was not designed to replace human feedback but to complement other professional development resources, she said.

Along with Dora Demszky, Jing Liu, and Chris Piech, the study was co-authored by Dan Jurafsky , a professor of linguistics and of computer science at Stanford, and Heather C. Hill, a professor at the Harvard Graduate School of Education.

Media Contacts

Dora Demszky, Stanford Graduate School of Education: [email protected]

Carrie Spector, Stanford Graduate School of Education: [email protected]

English for Academic Research: Grammar Exercises

• © 2024
• Latest edition
• Adrian Wallwork 0

Adrian Wallwork, Pisa, Italy

You can also search for this author in PubMed   Google Scholar

• Real-life examples from over 5000 papers written by non-native English speakers
• Book includes a key with detailed explanations
• Includes supplementary material: sn.pub/extras

Part of the book series: English for Academic Research (EAR)

181 Accesses

This is a preview of subscription content, log in via an institution to check access.

Access this book

• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

Table of contents (30 chapters)

Front matter, nouns: plurals, countable versus uncountable, etc.

Adrian Wallwork

Genitive: the possessive form of nouns

Indefinite article ( a / an ), definite article ( the ), and zero article (ø), quantifiers: some, any, little, few, a lot of, lots, much, many, relative pronouns: that, which, who, whose, what, present tenses, past tenses, future tenses, conditional forms: zero, first, second, third, mixed, passive versus active: impersonal versus personal forms, infinitive, - ing form (gerund), suggest, recommend, modal verbs, phrasal verbs, comparative and superlative forms, acronyms and abbreviations.

• English as Second Language
• Grammar Exercises
• Large Language Models
• Autotranslation

About this book

This book is based on a study of referees' reports and letters from journal editors on reasons why papers written by non-native researchers are rejected due to problems with English grammar.

The exercises include the following areas:

• active vs passive, use of we
• articles ( a/an, the , zero) and quantifiers ( some, any, few etc.)
• conditionals and modals
• countable and uncountable nouns
• infinitive vs -ing form
• numbers, acronyms, abbreviations
• relative clauses and which vs that
• tenses (e.g. simple present, simple past, present perfect)

This new edition includes exercises on using Large Language Models for generating and correcting emails, plus a separate chapter on using automatic translation.

The exercises can also be integrated into English for Academic Purposes (EAP) and English for Special Purposes (ESP) courses at universities and research institutes.

English for Research: Usage, Style, and Grammar English for Writing Research Papers English for Academic Correspondence and Socializing

Adrian Wallwork edits scientific papers and teaches English for Academic Purposes (EAP) to PhD students. In addition to his many books for Springer, he has written course books for Oxford University Press and discussion books for Cambridge University Press.

Authors and Affiliations

About the author.

Adrian Wallwork  edits scientific papers and teaches English for Academic Purposes (EAP) to PhD students. In addition to his many books for Springer, he has written course books for Oxford University Press and discussion books for Cambridge University Press.

Bibliographic Information

Book Title : English for Academic Research: Grammar Exercises

Authors : Adrian Wallwork

Series Title : English for Academic Research

DOI : https://doi.org/10.1007/978-3-031-53168-2

Publisher : Springer Cham

eBook Packages : Education , Education (R0)

Copyright Information : The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2024

Softcover ISBN : 978-3-031-53167-5 Published: 27 May 2024

eBook ISBN : 978-3-031-53168-2 Published: 26 May 2024

Series ISSN : 2625-3445

Series E-ISSN : 2625-3453

Edition Number : 2

Number of Pages : X, 232

Number of Illustrations : 3 b/w illustrations

Topics : Language Education , Linguistics, general

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Enter a Search Term

Experiential education in action: economics students (and faculty) present at prestigious research industry conference.

In March 2024, five Economics Department faculty and 10 undergraduate and graduate students, all Dyson economics and business economics majors, presented their research papers at the annual Eastern Economic Association (EEA) conference in Boston, one of the major conferences for professional economists on the East Coast of the United States.

The student presentations, organized by Assistant Professors Mary Kaltenberg, PhD and Eric Osborne Christenson, PhD , focused on timely topics such as labor economics, sports economics, applied microeconomics, and the savings and financial markets. Both faculty and students fielded many questions and comments from professional economists as well as other engaged attendees who took part in the well-attended sessions.

The participation in the EEA conference by students is a prime example of the many innovative, experiential education opportunities provided by the Economics Department (another being the College Federal Reserve Challenge , through which Pace’s team won the National College Federal Reserve Challenge five times in the last nine years, besting Harvard and Princeton). As part of their research leading up to the EEA conference, students had the opportunity to develop analytical and quantitative skills, while also building their portfolio and resume in a way that will stand out to employees, thus enhancing their competitive edge in today’s job market.

Professor and Chair of Economics Anna Shostya said, “Through original research and guidance from faculty mentors, our students gained the capacity to convert theoretical knowledge into practical use, bridging the gap between academic learning and real-world problem-solving.” 

Training in the (economic) trenches

McKenna Moore ’24, who is in the combined BA in Economics/MS in Applied Quantitative Economic Analysis and Policy program, and also a sports enthusiast, wanted to choose a topic that both interested her and was technically challenging, requiring the use and enhancement of the web scraping skills learned in R and Python classes taken at Pace.

“Through weekly meetings with Professor Kaltenberg, I was able to learn about different types of modeling that I never would have been exposed to in the classroom while pursuing my dual degree." —McKenna Moore ’24

On her experience researching the topic, An Examination of the “Loser Effect” in the NFL , she said, “Through weekly meetings with Professor Kaltenberg, I was able to learn about different types of modeling that I never would have been exposed to in the classroom while pursuing my dual degree. I was also able to lean on my peers who were simultaneously doing research to get their feedback as I progressed through this research, which was very helpful."

The comradery among students and their support of each other, in fact, did not go unnoticed by Kaltenberg, who was particularly impressed by the long hours they spent digging into each other’s research papers and their dedication to providing quality feedback. When asked about feedback provided by other attendees at the conference, she shares that many were surprised that students were at the undergraduate or even graduate level, and presenting original work.

“Many attendees assumed our students were doctoral students. I was proud of the level of their work–many used advanced econometric techniques, had clever topics and ideas, and presented professionally,” she said.

Liam Chentoufi ’25 is a BA in Economics major who, for the past two years, has also been a member of the College Federal Reserve Challenge team, which, just prior to the EEA conference, had met with the Board of Governors in Washington, DC (as well as Chairman Powell) to ask competition-specific questions to a panel of judges as a finale to the 2023 competitive season. Given this, Chentoufi was drawn to questions surrounding the Fed and the macroeconomy, and his EEA presentation topic was Understanding the Federal Reserve .

His main takeaway from the experience was learning more about how economics conferences are conducted and, relatedly, how he can further develop his research skills. He said, “In addition to presenting our own work, attendees are often asked to be a ‘discussant’ for other papers in which one presents their interpretation/questions for the peer paper assigned to them. After receiving audience feedback, I now have more organized ideas as to what can be improved upon in my personal project.”

“Many attendees assumed our students were doctoral students. I was proud of the level of their work–many used advanced econometric techniques, had clever topics and ideas, and presented professionally." —Assistant Professor of Economics Mary Kaltenberg, PhD

Macey Cooper ’24, who is also in the combined BA in Economics/MS in Applied Quantitative Economic Analysis and Policy program, had always been interested in the healthcare sector, and, when she was younger, thought she might even want to be a doctor. Her presentation topic, Medicaid and Maternal Mortality Rates , was therefore the perfect opportunity to combine her interest in both the medical field and in economics, as she chose to look at Medicaid expansion under Obamacare, and how it might affect pregnant women.

She shares how attending the EEA conference brought her out of my comfort zone as she was tasked with presenting in front of individuals outside of the Pace community. To use an economics term, her experience was very “macro:” “I was able to gain insight into how my research resonated with a broader community, and how it fits into the overall ecosystem of the economic impact on healthcare,” she said.

The importance of funding

Kaltenberg, who also presented research at the conference with colleague, Eric Osborne Christenson, PhD, on the effect of a lack of childcare access due to Covid-19 on female labor force participation, shared that many students had been motivated to continue a career in research or consider a graduate degree in economics due to their experiences.

It was all made possible by the Dyson Initiative Fund, which allows students to participate annually, versus every two to three years, as is the case with the New York City conference.

“This funding is crucial to allow every generation of economics majors the opportunity to present their work in a professional environment. We hope to send our best students who do original research to this conference every year,” said Kaltenberg.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 17 October 2023

The impact of founder personalities on startup success

• Paul X. McCarthy 1 , 2 ,
• Xian Gong 3 ,
• Fabian Braesemann 4 , 5 ,
• Fabian Stephany 4 , 5 ,
• Marian-Andrei Rizoiu 3 &
• Margaret L. Kern 6  

Scientific Reports volume  13 , Article number:  17200 ( 2023 ) Cite this article

59k Accesses

2 Citations

305 Altmetric

Metrics details

• Human behaviour
• Information technology

An Author Correction to this article was published on 07 May 2024

This article has been updated

Startup companies solve many of today’s most challenging problems, such as the decarbonisation of the economy or the development of novel life-saving vaccines. Startups are a vital source of innovation, yet the most innovative are also the least likely to survive. The probability of success of startups has been shown to relate to several firm-level factors such as industry, location and the economy of the day. Still, attention has increasingly considered internal factors relating to the firm’s founding team, including their previous experiences and failures, their centrality in a global network of other founders and investors, as well as the team’s size. The effects of founders’ personalities on the success of new ventures are, however, mainly unknown. Here, we show that founder personality traits are a significant feature of a firm’s ultimate success. We draw upon detailed data about the success of a large-scale global sample of startups (n = 21,187). We find that the Big Five personality traits of startup founders across 30 dimensions significantly differ from that of the population at large. Key personality facets that distinguish successful entrepreneurs include a preference for variety, novelty and starting new things (openness to adventure), like being the centre of attention (lower levels of modesty) and being exuberant (higher activity levels). We do not find one ’Founder-type’ personality; instead, six different personality types appear. Our results also demonstrate the benefits of larger, personality-diverse teams in startups, which show an increased likelihood of success. The findings emphasise the role of the diversity of personality types as a novel dimension of team diversity that influences performance and success.

Similar content being viewed by others

Predicting success in the worldwide start-up network

The personality traits of self-made and inherited millionaires

The nexus of top executives’ attributes, firm strategies, and outcomes: Large firms versus SMEs

Introduction.

The success of startups is vital to economic growth and renewal, with a small number of young, high-growth firms creating a disproportionately large share of all new jobs 1 , 2 . Startups create jobs and drive economic growth, and they are also an essential vehicle for solving some of society’s most pressing challenges.

As a poignant example, six centuries ago, the German city of Mainz was abuzz as the birthplace of the world’s first moveable-type press created by Johannes Gutenberg. However, in the early part of this century, it faced several economic challenges, including rising unemployment and a significant and growing municipal debt. Then in 2008, two Turkish immigrants formed the company BioNTech in Mainz with another university research colleague. Together they pioneered new mRNA-based technologies. In 2020, BioNTech partnered with US pharmaceutical giant Pfizer to create one of only a handful of vaccines worldwide for Covid-19, saving an estimated six million lives 3 . The economic benefit to Europe and, in particular, the German city where the vaccine was developed has been significant, with windfall tax receipts to the government clearing Mainz’s €1.3bn debt and enabling tax rates to be reduced, attracting other businesses to the region as well as inspiring a whole new generation of startups 4 .

While stories such as the success of BioNTech are often retold and remembered, their success is the exception rather than the rule. The overwhelming majority of startups ultimately fail. One study of 775 startups in Canada that successfully attracted external investment found only 35% were still operating seven years later 5 .

But what determines the success of these ‘lucky few’? When assessing the success factors of startups, especially in the early-stage unproven phase, venture capitalists and other investors offer valuable insights. Three different schools of thought characterise their perspectives: first, supply-side or product investors : those who prioritise investing in firms they consider to have novel and superior products and services, investing in companies with intellectual property such as patents and trademarks. Secondly, demand-side or market-based investors : those who prioritise investing in areas of highest market interest, such as in hot areas of technology like quantum computing or recurrent or emerging large-scale social and economic challenges such as the decarbonisation of the economy. Thirdly, talent investors : those who prioritise the foundation team above the startup’s initial products or what industry or problem it is looking to address.

Investors who adopt the third perspective and prioritise talent often recognise that a good team can overcome many challenges in the lead-up to product-market fit. And while the initial products of a startup may or may not work a successful and well-functioning team has the potential to pivot to new markets and new products, even if the initial ones prove untenable. Not surprisingly, an industry ‘autopsy’ into 101 tech startup failures found 23% were due to not having the right team—the number three cause of failure ahead of running out of cash or not having a product that meets the market need 6 .

Accordingly, early entrepreneurship research was focused on the personality of founders, but the focus shifted away in the mid-1980s onwards towards more environmental factors such as venture capital financing 7 , 8 , 9 , networks 10 , location 11 and due to a range of issues and challenges identified with the early entrepreneurship personality research 12 , 13 . At the turn of the 21st century, some scholars began exploring ways to combine context and personality and reconcile entrepreneurs’ individual traits with features of their environment. In her influential work ’The Sociology of Entrepreneurship’, Patricia H. Thornton 14 discusses two perspectives on entrepreneurship: the supply-side perspective (personality theory) and the demand-side perspective (environmental approach). The supply-side perspective focuses on the individual traits of entrepreneurs. In contrast, the demand-side perspective focuses on the context in which entrepreneurship occurs, with factors such as finance, industry and geography each playing their part. In the past two decades, there has been a revival of interest and research that explores how entrepreneurs’ personality relates to the success of their ventures. This new and growing body of research includes several reviews and meta-studies, which show that personality traits play an important role in both career success and entrepreneurship 15 , 16 , 17 , 18 , 19 , that there is heterogeneity in definitions and samples used in research on entrepreneurship 16 , 18 , and that founder personality plays an important role in overall startup outcomes 17 , 19 .

Motivated by the pivotal role of the personality of founders on startup success outlined in these recent contributions, we investigate two main research questions:

Which personality features characterise founders?

Do their personalities, particularly the diversity of personality types in founder teams, play a role in startup success?

We aim to understand whether certain founder personalities and their combinations relate to startup success, defined as whether their company has been acquired, acquired another company or listed on a public stock exchange. For the quantitative analysis, we draw on a previously published methodology 20 , which matches people to their ‘ideal’ jobs based on social media-inferred personality traits.

We find that personality traits matter for startup success. In addition to firm-level factors of location, industry and company age, we show that founders’ specific Big Five personality traits, such as adventurousness and openness, are significantly more widespread among successful startups. As we find that companies with multi-founder teams are more likely to succeed, we cluster founders in six different and distinct personality groups to underline the relevance of the complementarity in personality traits among founder teams. Startups with diverse and specific combinations of founder types (e. g., an adventurous ‘Leader’, a conscientious ‘Accomplisher’, and an extroverted ‘Developer’) have significantly higher odds of success.

We organise the rest of this paper as follows. In the Section " Results ", we introduce the data used and the methods applied to relate founders’ psychological traits with their startups’ success. We introduce the natural language processing method to derive individual and team personality characteristics and the clustering technique to identify personality groups. Then, we present the result for multi-variate regression analysis that allows us to relate firm success with external and personality features. Subsequently, the Section " Discussion " mentions limitations and opportunities for future research in this domain. In the Section " Methods ", we describe the data, the variables in use, and the clustering in greater detail. Robustness checks and additional analyses can be found in the Supplementary Information.

Our analysis relies on two datasets. We infer individual personality facets via a previously published methodology 20 from Twitter user profiles. Here, we restrict our analysis to founders with a Crunchbase profile. Crunchbase is the world’s largest directory on startups. It provides information about more than one million companies, primarily focused on funding and investors. A company’s public Crunchbase profile can be considered a digital business card of an early-stage venture. As such, the founding teams tend to provide information about themselves, including their educational background or a link to their Twitter account.

We infer the personality profiles of the founding teams of early-stage ventures from their publicly available Twitter profiles, using the methodology described by Kern et al. 20 . Then, we correlate this information to data from Crunchbase to determine whether particular combinations of personality traits correspond to the success of early-stage ventures. The final dataset used in the success prediction model contains n = 21,187 startup companies (for more details on the data see the Methods section and SI section  A.5 ).

Revisions of Crunchbase as a data source for investigations on a firm and industry level confirm the platform to be a useful and valuable source of data for startups research, as comparisons with other sources at micro-level, e.g., VentureXpert or PwC, also suggest that the platform’s coverage is very comprehensive, especially for start-ups located in the United States 21 . Moreover, aggregate statistics on funding rounds by country and year are quite similar to those produced with other established sources, going to validate the use of Crunchbase as a reliable source in terms of coverage of funded ventures. For instance, Crunchbase covers about the same number of investment rounds in the analogous sectors as collected by the National Venture Capital Association 22 . However, we acknowledge that the data source might suffer from registration latency (a certain delay between the foundation of the company and its actual registration on Crunchbase) and success bias in company status (the likeliness that failed companies decide to delete their profile from the database).

The definition of startup success

The success of startups is uncertain, dependent on many factors and can be measured in various ways. Due to the likelihood of failure in startups, some large-scale studies have looked at which features predict startup survival rates 23 , and others focus on fundraising from external investors at various stages 24 . Success for startups can be measured in multiple ways, such as the amount of external investment attracted, the number of new products shipped or the annual growth in revenue. But sometimes external investments are misguided, revenue growth can be short-lived, and new products may fail to find traction.

Success in a startup is typically staged and can appear in different forms and times. For example, a startup may be seen to be successful when it finds a clear solution to a widely recognised problem, such as developing a successful vaccine. On the other hand, it could be achieving some measure of commercial success, such as rapidly accelerating sales or becoming profitable or at least cash positive. Or it could be reaching an exit for foundation investors via a trade sale, acquisition or listing of its shares for sale on a public stock exchange via an Initial Public Offering (IPO).

For our study, we focused on the startup’s extrinsic success rather than the founders’ intrinsic success per se, as its more visible, objective and measurable. A frequently considered measure of success is the attraction of external investment by venture capitalists 25 . However, this is not in and of itself a good measure of clear, incontrovertible success, particularly for early-stage ventures. This is because it reflects investors’ expectations of a startup’s success potential rather than actual business success. Similarly, we considered other measures like revenue growth 26 , liquidity events 27 , 28 , 29 , profitability 30 and social impact 31 , all of which have benefits as they capture incremental success, but each also comes with operational measurement challenges.

Therefore, we apply the success definition initially introduced by Bonaventura et al. 32 , namely that a startup is acquired, acquires another company or has an initial public offering (IPO). We consider any of these major capital liquidation events as a clear threshold signal that the company has matured from an early-stage venture to becoming or is on its way to becoming a mature company with clear and often significant business growth prospects. Together these three major liquidity events capture the primary forms of exit for external investors (an acquisition or trade sale and an IPO). For companies with a longer autonomous growth runway, acquiring another company marks a similar milestone of scale, maturity and capability.

Using multifactor analysis and a binary classification prediction model of startup success, we looked at many variables together and their relative influence on the probability of the success of startups. We looked at seven categories of factors through three lenses of firm-level factors: (1) location, (2) industry, (3) age of the startup; founder-level factors: (4) number of founders, (5) gender of founders, (6) personality characteristics of founders and; lastly team-level factors: (7) founder-team personality combinations. The model performance and relative impacts on the probability of startup success of each of these categories of founders are illustrated in more detail in section  A.6 of the Supplementary Information (in particular Extended Data Fig.  19 and Extended Data Fig.  20 ). In total, we considered over three hundred variables (n = 323) and their relative significant associations with success.

The personality of founders

Besides product-market, industry, and firm-level factors (see SI section  A.1 ), research suggests that the personalities of founders play a crucial role in startup success 19 . Therefore, we examine the personality characteristics of individual startup founders and teams of founders in relationship to their firm’s success by applying the success definition used by Bonaventura et al. 32 .

Employing established methods 33 , 34 , 35 , we inferred the personality traits across 30 dimensions (Big Five facets) of a large global sample of startup founders. The startup founders cohort was created from a subset of founders from the global startup industry directory Crunchbase, who are also active on the social media platform Twitter.

To measure the personality of the founders, we used the Big Five, a popular model of personality which includes five core traits: Openness to Experience, Conscientiousness, Extraversion, Agreeableness, and Emotional stability. Each of these traits can be further broken down into thirty distinct facets. Studies have found that the Big Five predict meaningful life outcomes, such as physical and mental health, longevity, social relationships, health-related behaviours, antisocial behaviour, and social contribution, at levels on par with intelligence and socioeconomic status 36 Using machine learning to infer personality traits by analysing the use of language and activity on social media has been shown to be more accurate than predictions of coworkers, friends and family and similar in accuracy to the judgement of spouses 37 . Further, as other research has shown, we assume that personality traits remain stable in adulthood even through significant life events 38 , 39 , 40 . Personality traits have been shown to emerge continuously from those already evident in adolescence 41 and are not significantly influenced by external life events such as becoming divorced or unemployed 42 . This suggests that the direction of any measurable effect goes from founder personalities to startup success and not vice versa.

As a first investigation to what extent personality traits might relate to entrepreneurship, we use the personality characteristics of individuals to predict whether they were an entrepreneur or an employee. We trained and tested a machine-learning random forest classifier to distinguish and classify entrepreneurs from employees and vice-versa using inferred personality vectors alone. As a result, we found we could correctly predict entrepreneurs with 77% accuracy and employees with 88% accuracy (Fig.  1 A). Thus, based on personality information alone, we correctly predict all unseen new samples with 82.5% accuracy (See SI section  A.2 for more details on this analysis, the classification modelling and prediction accuracy).

We explored in greater detail which personality features are most prominent among entrepreneurs. We found that the subdomain or facet of Adventurousness within the Big Five Domain of Openness was significant and had the largest effect size. The facet of Modesty within the Big Five Domain of Agreeableness and Activity Level within the Big Five Domain of Extraversion was the subsequent most considerable effect (Fig.  1 B). Adventurousness in the Big Five framework is defined as the preference for variety, novelty and starting new things—which are consistent with the role of a startup founder whose role, especially in the early life of the company, is to explore things that do not scale easily 43 and is about developing and testing new products, services and business models with the market.

Once we derived and tested the Big Five personality features for each entrepreneur in our data set, we examined whether there is evidence indicating that startup founders naturally cluster according to their personality features using a Hopkins test (see Extended Data Figure  6 ). We discovered clear clustering tendencies in the data compared with other renowned reference data sets known to have clusters. Then, once we established the founder data clusters, we used agglomerative hierarchical clustering. This ‘bottom-up’ clustering technique initially treats each observation as an individual cluster. Then it merges them to create a hierarchy of possible cluster schemes with differing numbers of groups (See Extended Data Fig.  7 ). And lastly, we identified the optimum number of clusters based on the outcome of four different clustering performance measurements: Davies-Bouldin Index, Silhouette coefficients, Calinski-Harabas Index and Dunn Index (see Extended Data Figure  8 ). We find that the optimum number of clusters of startup founders based on their personality features is six (labelled #0 through to #5), as shown in Fig.  1 C.

To better understand the context of different founder types, we positioned each of the six types of founders within an occupation-personality matrix established from previous research 44 . This research showed that ‘each job has its own personality’ using a substantial sample of employees across various jobs. Utilising the methodology employed in this study, we assigned labels to the cluster names #0 to #5, which correspond to the identified occupation tribes that best describe the personality facets represented by the clusters (see Extended Data Fig.  9 for an overview of these tribes, as identified by McCarthy et al. 44 ).

Utilising this approach, we identify three ’purebred’ clusters: #0, #2 and #5, whose members are dominated by a single tribe (larger than 60% of all individuals in each cluster are characterised by one tribe). Thus, these clusters represent and share personality attributes of these previously identified occupation-personality tribes 44 , which have the following known distinctive personality attributes (see also Table  1 ):

Accomplishers (#0) —Organised & outgoing. confident, down-to-earth, content, accommodating, mild-tempered & self-assured.

Leaders (#2) —Adventurous, persistent, dispassionate, assertive, self-controlled, calm under pressure, philosophical, excitement-seeking & confident.

Fighters (#5) —Spontaneous and impulsive, tough, sceptical, and uncompromising.

We labelled these clusters with the tribe names, acknowledging that labels are somewhat arbitrary, based on our best interpretation of the data (See SI section  A.3 for more details).

For the remaining three clusters #1, #3 and #4, we can see they are ‘hybrids’, meaning that the founders within them come from a mix of different tribes, with no one tribe representing more than 50% of the members of that cluster. However, the tribes with the largest share were noted as #1 Experts/Engineers, #3 Fighters, and #4 Operators.

To label these three hybrid clusters, we examined the closest occupations to the median personality features of each cluster. We selected a name that reflected the common themes of these occupations, namely:

Experts/Engineers (#1) as the closest roles included Materials Engineers and Chemical Engineers. This is consistent with this cluster’s personality footprint, which is highest in openness in the facets of imagination and intellect.

Developers (#3) as the closest roles include Application Developers and related technology roles such as Business Systems Analysts and Product Managers.

Operators (#4) as the closest roles include service, maintenance and operations functions, including Bicycle Mechanic, Mechanic and Service Manager. This is also consistent with one of the key personality traits of high conscientiousness in the facet of orderliness and high agreeableness in the facet of humility for founders in this cluster.

Founder-Level Factors of Startup Success. ( A ), Successful entrepreneurs differ from successful employees. They can be accurately distinguished using a classifier with personality information alone. ( B ), Successful entrepreneurs have different Big Five facet distributions, especially on adventurousness, modesty and activity level. ( C ), Founders come in six different types: Fighters, Operators, Accomplishers, Leaders, Engineers and Developers (FOALED) ( D ), Each founder Personality-Type has its distinct facet.

Together, these six different types of startup founders (Fig.  1 C) represent a framework we call the FOALED model of founder types—an acronym of Fighters, Operators, Accomplishers, Leaders, Engineers and D evelopers.

Each founder’s personality type has its distinct facet footprint (for more details, see Extended Data Figure  10 in SI section  A.3 ). Also, we observe a central core of correlated features that are high for all types of entrepreneurs, including intellect, adventurousness and activity level (Fig.  1 D).To test the robustness of the clustering of the personality facets, we compare the mean scores of the individual facets per cluster with a 20-fold resampling of the data and find that the clusters are, overall, largely robust against resampling (see Extended Data Figure  11 in SI section  A.3 for more details).

We also find that the clusters accord with the distribution of founders’ roles in their startups. For example, Accomplishers are often Chief Executive Officers, Chief Financial Officers, or Chief Operating Officers, while Fighters tend to be Chief Technical Officers, Chief Product Officers, or Chief Commercial Officers (see Extended Data Fig.  12 in SI section  A.4 for more details).

The ensemble theory of success

While founders’ individual personality traits, such as Adventurousness or Openness, show to be related to their firms’ success, we also hypothesise that the combination, or ensemble, of personality characteristics of a founding team impacts the chances of success. The logic behind this reasoning is complementarity, which is proposed by contemporary research on the functional roles of founder teams. Examples of these clear functional roles have evolved in established industries such as film and television, construction, and advertising 45 . When we subsequently explored the combinations of personality types among founders and their relationship to the probability of startup success, adjusted for a range of other factors in a multi-factorial analysis, we found significantly increased chances of success for mixed foundation teams:

Initially, we find that firms with multiple founders are more likely to succeed, as illustrated in Fig.  2 A, which shows firms with three or more founders are more than twice as likely to succeed than solo-founded startups. This finding is consistent with investors’ advice to founders and previous studies 46 . We also noted that some personality types of founders increase the probability of success more than others, as shown in SI section  A.6 (Extended Data Figures  16 and 17 ). Also, we note that gender differences play out in the distribution of personality facets: successful female founders and successful male founders show facet scores that are more similar to each other than are non-successful female founders to non-successful male founders (see Extended Data Figure  18 ).

The Ensemble Theory of Team-Level Factors of Startup Success. ( A ) Having a larger founder team elevates the chances of success. This can be due to multiple reasons, e.g., a more extensive network or knowledge base but also personality diversity. ( B ) We show that joint personality combinations of founders are significantly related to higher chances of success. This is because it takes more than one founder to cover all beneficial personality traits that ‘breed’ success. ( C ) In our multifactor model, we show that firms with diverse and specific combinations of types of founders have significantly higher odds of success.

Access to more extensive networks and capital could explain the benefits of having more founders. Still, as we find here, it also offers a greater diversity of combined personalities, naturally providing a broader range of maximum traits. So, for example, one founder may be more open and adventurous, and another could be highly agreeable and trustworthy, thus, potentially complementing each other’s particular strengths associated with startup success.

The benefits of larger and more personality-diverse foundation teams can be seen in the apparent differences between successful and unsuccessful firms based on their combined Big Five personality team footprints, as illustrated in Fig.  2 B. Here, maximum values for each Big Five trait of a startup’s co-founders are mapped; stratified by successful and non-successful companies. Founder teams of successful startups tend to score higher on Openness, Conscientiousness, Extraversion, and Agreeableness.

When examining the combinations of founders with different personality types, we find that some ensembles of personalities were significantly correlated with greater chances of startup success—while controlling for other variables in the model—as shown in Fig.  2 C (for more details on the modelling, the predictive performance and the coefficient estimates of the final model, see Extended Data Figures  19 , 20 , and 21 in SI section  A.6 ).

Three combinations of trio-founder companies were more than twice as likely to succeed than other combinations, namely teams with (1) a Leader and two Developers , (2) an Operator and two Developers , and (3) an Expert/Engineer , Leader and Developer . To illustrate the potential mechanisms on how personality traits might influence the success of startups, we provide some examples of well-known, successful startup founders and their characteristic personality traits in Extended Data Figure  22 .

Startups are one of the key mechanisms for brilliant ideas to become solutions to some of the world’s most challenging economic and social problems. Examples include the Google search algorithm, disability technology startup Fingerwork’s touchscreen technology that became the basis of the Apple iPhone, or the Biontech mRNA technology that powered Pfizer’s COVID-19 vaccine.

We have shown that founders’ personalities and the combination of personalities in the founding team of a startup have a material and significant impact on its likelihood of success. We have also shown that successful startup founders’ personality traits are significantly different from those of successful employees—so much so that a simple predictor can be trained to distinguish between employees and entrepreneurs with more than 80% accuracy using personality trait data alone.

Just as occupation-personality maps derived from data can provide career guidance tools, so too can data on successful entrepreneurs’ personality traits help people decide whether becoming a founder may be a good choice for them.

We have learnt through this research that there is not one type of ideal ’entrepreneurial’ personality but six different types. Many successful startups have multiple co-founders with a combination of these different personality types.

To a large extent, founding a startup is a team sport; therefore, diversity and complementarity of personalities matter in the foundation team. It has an outsized impact on the company’s likelihood of success. While all startups are high risk, the risk becomes lower with more founders, particularly if they have distinct personality traits.

Our work demonstrates the benefits of personality diversity among the founding team of startups. Greater awareness of this novel form of diversity may help create more resilient startups capable of more significant innovation and impact.

The data-driven research approach presented here comes with certain methodological limitations. The principal data sources of this study—Crunchbase and Twitter—are extensive and comprehensive, but there are characterised by some known and likely sample biases.

Crunchbase is the principal public chronicle of venture capital funding. So, there is some likely sample bias toward: (1) Startup companies that are funded externally: self-funded or bootstrapped companies are less likely to be represented in Crunchbase; (2) technology companies, as that is Crunchbase’s roots; (3) multi-founder companies; (4) male founders: while the representation of female founders is now double that of the mid-2000s, women still represent less than 25% of the sample; (5) companies that succeed: companies that fail, especially those that fail early, are likely to be less represented in the data.

Samples were also limited to those founders who are active on Twitter, which adds additional selection biases. For example, Twitter users typically are younger, more educated and have a higher median income 47 . Another limitation of our approach is the potentially biased presentation of a person’s digital identity on social media, which is the basis for identifying personality traits. For example, recent research suggests that the language and emotional tone used by entrepreneurs in social media can be affected by events such as business failure 48 , which might complicate the personality trait inference.

In addition to sampling biases within the data, there are also significant historical biases in startup culture. For many aspects of the entrepreneurship ecosystem, women, for example, are at a disadvantage 49 . Male-founded companies have historically dominated most startup ecosystems worldwide, representing the majority of founders and the overwhelming majority of venture capital investors. As a result, startups with women have historically attracted significantly fewer funds 50 , in part due to the male bias among venture investors, although this is now changing, albeit slowly 51 .

The research presented here provides quantitative evidence for the relevance of personality types and the diversity of personalities in startups. At the same time, it brings up other questions on how personality traits are related to other factors associated with success, such as:

Will the recent growing focus on promoting and investing in female founders change the nature, composition and dynamics of startups and their personalities leading to a more diverse personality landscape in startups?

Will the growth of startups outside of the United States change what success looks like to investors and hence the role of different personality traits and their association to diverse success metrics?

Many of today’s most renowned entrepreneurs are either Baby Boomers (such as Gates, Branson, Bloomberg) or Generation Xers (such as Benioff, Cannon-Brookes, Musk). However, as we can see, personality is both a predictor and driver of success in entrepreneurship. Will generation-wide differences in personality and outlook affect startups and their success?

Moreover, the findings shown here have natural extensions and applications beyond startups, such as for new projects within large established companies. While not technically startups, many large enterprises and industries such as construction, engineering and the film industry rely on forming new project-based, cross-functional teams that are often new ventures and share many characteristics of startups.

There is also potential for extending this research in other settings in government, NGOs, and within the research community. In scientific research, for example, team diversity in terms of age, ethnicity and gender has been shown to be predictive of impact, and personality diversity may be another critical dimension 52 .

Another extension of the study could investigate the development of the language used by startup founders on social media over time. Such an extension could investigate whether the language (and inferred psychological characteristics) change as the entrepreneurs’ ventures go through major business events such as foundation, funding, or exit.

Overall, this study demonstrates, first, that startup founders have significantly different personalities than employees. Secondly, besides firm-level factors, which are known to influence firm success, we show that a range of founder-level factors, notably the character traits of its founders, significantly impact a startup’s likelihood of success. Lastly, we looked at team-level factors. We discovered in a multifactor analysis that personality-diverse teams have the most considerable impact on the probability of a startup’s success, underlining the importance of personality diversity as a relevant factor of team performance and success.

Data sources

Entrepreneurs dataset.

Data about the founders of startups were collected from Crunchbase (Table  2 ), an open reference platform for business information about private and public companies, primarily early-stage startups. It is one of the largest and most comprehensive data sets of its kind and has been used in over 100 peer-reviewed research articles about economic and managerial research.

Crunchbase contains data on over two million companies - mainly startup companies and the companies who partner with them, acquire them and invest in them, as well as profiles on well over one million individuals active in the entrepreneurial ecosystem worldwide from over 200 countries and spans. Crunchbase started in the technology startup space, and it now covers all sectors, specifically focusing on entrepreneurship, investment and high-growth companies.

While Crunchbase contains data on over one million individuals in the entrepreneurial ecosystem, some are not entrepreneurs or startup founders but play other roles, such as investors, lawyers or executives at companies that acquire startups. To create a subset of only entrepreneurs, we selected a subset of 32,732 who self-identify as founders and co-founders (by job title) and who are also publicly active on the social media platform Twitter. We also removed those who also are venture capitalists to distinguish between investors and founders.

We selected founders active on Twitter to be able to use natural language processing to infer their Big Five personality features using an open-vocabulary approach shown to be accurate in the previous research by analysing users’ unstructured text, such as Twitter posts in our case. For this project, as with previous research 20 , we employed a commercial service, IBM Watson Personality Insight, to infer personality facets. This service provides raw scores and percentile scores of Big Five Domains (Openness, Conscientiousness, Extraversion, Agreeableness and Emotional Stability) and the corresponding 30 subdomains or facets. In addition, the public content of Twitter posts was collected, and there are 32,732 profiles that each had enough Twitter posts (more than 150 words) to get relatively accurate personality scores (less than 12.7% Average Mean Absolute Error).

The entrepreneurs’ dataset is analysed in combination with other data about the companies they founded to explore questions about the nature and patterns of personality traits of entrepreneurs and the relationships between these patterns and company success.

For the multifactor analysis, we further filtered the data in several preparatory steps for the success prediction modelling (for more details, see SI section  A.5 ). In particular, we removed data points with missing values (Extended Data Fig.  13 ) and kept only companies in the data that were founded from 1990 onward to ensure consistency with previous research 32 (see Extended Data Fig.  14 ). After cleaning, filtering and pre-processing the data, we ended up with data from 25,214 founders who founded 21,187 startup companies to be used in the multifactor analysis. Of those, 3442 startups in the data were successful, 2362 in the first seven years after they were founded (see Extended Data Figure  15 for more details).

Entrepreneurs and employees dataset

To investigate whether startup founders show personality traits that are similar or different from the population at large (i. e. the entrepreneurs vs employees sub-analysis shown in Fig.  1 A and B), we filtered the entrepreneurs’ data further: we reduced the sample to those founders of companies, which attracted more than US$100k in investment to create a reference set of successful entrepreneurs (n \(=\) 4400).

To create a control group of employees who are not also entrepreneurs or very unlikely to be of have been entrepreneurs, we leveraged the fact that while some occupational titles like CEO, CTO and Public Speaker are commonly shared by founders and co-founders, some others such as Cashier , Zoologist and Detective very rarely co-occur seem to be founders or co-founders. To illustrate, many company founders also adopt regular occupation titles such as CEO or CTO. Many founders will be Founder and CEO or Co-founder and CTO. While founders are often CEOs or CTOs, the reverse is not necessarily true, as many CEOs are professional executives that were not involved in the establishment or ownership of the firm.

Using data from LinkedIn, we created an Entrepreneurial Occupation Index (EOI) based on the ratio of entrepreneurs for each of the 624 occupations used in a previous study of occupation-personality fit 44 . It was calculated based on the percentage of all people working in the occupation from LinkedIn compared to those who shared the title Founder or Co-founder (See SI section  A.2 for more details). A reference set of employees (n=6685) was then selected across the 112 different occupations with the lowest propensity for entrepreneurship (less than 0.5% EOI) from a large corpus of Twitter users with known occupations, which is also drawn from the previous occupational-personality fit study 44 .

These two data sets were used to test whether it may be possible to distinguish successful entrepreneurs from successful employees based on the different patterns of personality traits alone.

Hierarchical clustering

We applied several clustering techniques and tests to the personality vectors of the entrepreneurs’ data set to determine if there are natural clusters and, if so, how many are the optimum number.

Firstly, to determine if there is a natural typology to founder personalities, we applied the Hopkins statistic—a statistical test we used to answer whether the entrepreneurs’ dataset contains inherent clusters. It measures the clustering tendency based on the ratio of the sum of distances of real points within a sample of the entrepreneurs’ dataset to their nearest neighbours and the sum of distances of randomly selected artificial points from a simulated uniform distribution to their nearest neighbours in the real entrepreneurs’ dataset. The ratio measures the difference between the entrepreneurs’ data distribution and the simulated uniform distribution, which tests the randomness of the data. The range of Hopkins statistics is from 0 to 1. The scores are close to 0, 0.5 and 1, respectively, indicating whether the dataset is uniformly distributed, randomly distributed or highly clustered.

To cluster the founders by personality facets, we used Agglomerative Hierarchical Clustering (AHC)—a bottom-up approach that treats an individual data point as a singleton cluster and then iteratively merges pairs of clusters until all data points are included in the single big collection. Ward’s linkage method is used to choose the pair of groups for minimising the increase in the within-cluster variance after combining. AHC was widely applied to clustering analysis since a tree hierarchy output is more informative and interpretable than K-means. Dendrograms were used to visualise the hierarchy to provide the perspective of the optimal number of clusters. The heights of the dendrogram represent the distance between groups, with lower heights representing more similar groups of observations. A horizontal line through the dendrogram was drawn to distinguish the number of significantly different clusters with higher heights. However, as it is not possible to determine the optimum number of clusters from the dendrogram, we applied other clustering performance metrics to analyse the optimal number of groups.

A range of Clustering performance metrics were used to help determine the optimal number of clusters in the dataset after an apparent clustering tendency was confirmed. The following metrics were implemented to evaluate the differences between within-cluster and between-cluster distances comprehensively: Dunn Index, Calinski-Harabasz Index, Davies-Bouldin Index and Silhouette Index. The Dunn Index measures the ratio of the minimum inter-cluster separation and the maximum intra-cluster diameter. At the same time, the Calinski-Harabasz Index improves the measurement of the Dunn Index by calculating the ratio of the average sum of squared dispersion of inter-cluster and intra-cluster. The Davies-Bouldin Index simplifies the process by treating each cluster individually. It compares the sum of the average distance among intra-cluster data points to the cluster centre of two separate groups with the distance between their centre points. Finally, the Silhouette Index is the overall average of the silhouette coefficients for each sample. The coefficient measures the similarity of the data point to its cluster compared with the other groups. Higher scores of the Dunn, Calinski-Harabasz and Silhouette Index and a lower score of the Davies-Bouldin Index indicate better clustering configuration.

Classification modelling

Classification algorithms.

To obtain a comprehensive and robust conclusion in the analysis predicting whether a given set of personality traits corresponds to an entrepreneur or an employee, we explored the following classifiers: Naïve Bayes, Elastic Net regularisation, Support Vector Machine, Random Forest, Gradient Boosting and Stacked Ensemble. The Naïve Bayes classifier is a probabilistic algorithm based on Bayes’ theorem with assumptions of independent features and equiprobable classes. Compared with other more complex classifiers, it saves computing time for large datasets and performs better if the assumptions hold. However, in the real world, those assumptions are generally violated. Elastic Net regularisation combines the penalties of Lasso and Ridge to regularise the Logistic classifier. It eliminates the limitation of multicollinearity in the Lasso method and improves the limitation of feature selection in the Ridge method. Even though Elastic Net is as simple as the Naïve Bayes classifier, it is more time-consuming. The Support Vector Machine (SVM) aims to find the ideal line or hyperplane to separate successful entrepreneurs and employees in this study. The dividing line can be non-linear based on a non-linear kernel, such as the Radial Basis Function Kernel. Therefore, it performs well on high-dimensional data while the ’right’ kernel selection needs to be tuned. Random Forest (RF) and Gradient Boosting Trees (GBT) are ensembles of decision trees. All trees are trained independently and simultaneously in RF, while a new tree is trained each time and corrected by previously trained trees in GBT. RF is a more robust and straightforward model since it does not have many hyperparameters to tune. GBT optimises the objective function and learns a more accurate model since there is a successive learning and correction process. Stacked Ensemble combines all existing classifiers through a Logistic Regression. Better than bagging with only variance reduction and boosting with only bias reduction, the ensemble leverages the benefit of model diversity with both lower variance and bias. All the above classification algorithms distinguish successful entrepreneurs and employees based on the personality matrix.

Evaluation metrics

A range of evaluation metrics comprehensively explains the performance of a classification prediction. The most straightforward metric is accuracy, which measures the overall portion of correct predictions. It will mislead the performance of an imbalanced dataset. The F1 score is better than accuracy by combining precision and recall and considering the False Negatives and False Positives. Specificity measures the proportion of detecting the true negative rate that correctly identifies employees, while Positive Predictive Value (PPV) calculates the probability of accurately predicting successful entrepreneurs. Area Under the Receiver Operating Characteristic Curve (AUROC) determines the capability of the algorithm to distinguish between successful entrepreneurs and employees. A higher value means the classifier performs better on separating the classes.

Feature importance

To further understand and interpret the classifier, it is critical to identify variables with significant predictive power on the target. Feature importance of tree-based models measures Gini importance scores for all predictors, which evaluate the overall impact of the model after cutting off the specific feature. The measurements consider all interactions among features. However, it does not provide insights into the directions of impacts since the importance only indicates the ability to distinguish different classes.

Statistical analysis

T-test, Cohen’s D and two-sample Kolmogorov-Smirnov test are introduced to explore how the mean values and distributions of personality facets between entrepreneurs and employees differ. The T-test is applied to determine whether the mean of personality facets of two group samples are significantly different from one another or not. The facets with significant differences detected by the hypothesis testing are critical to separate the two groups. Cohen’s d is to measure the effect size of the results of the previous t-test, which is the ratio of the mean difference to the pooled standard deviation. A larger Cohen’s d score indicates that the mean difference is greater than the variability of the whole sample. Moreover, it is interesting to check whether the two groups’ personality facets’ probability distributions are from the same distribution through the two-sample Kolmogorov-Smirnov test. There is no assumption about the distributions, but the test is sensitive to deviations near the centre rather than the tail.

Privacy and ethics

The focus of this research is to provide high-level insights about groups of startups, founders and types of founder teams rather than on specific individuals or companies. While we used unit record data from the publicly available data of company profiles from Crunchbase , we removed all identifiers from the underlying data on individual companies and founders and generated aggregate results, which formed the basis for our analysis and conclusions.

Data availability

A dataset which includes only aggregated statistics about the success of startups and the factors that influence is released as part of this research. Underlying data for all figures and the code to reproduce them are available on GitHub: https://github.com/Braesemann/FounderPersonalities . Please contact Fabian Braesemann ( [email protected] ) in case you have any further questions.

Change history

07 may 2024.

A Correction to this paper has been published: https://doi.org/10.1038/s41598-024-61082-7

Henrekson, M. & Johansson, D. Gazelles as job creators: A survey and interpretation of the evidence. Small Bus. Econ. 35 , 227–244 (2010).

Article   Google Scholar  

Davila, A., Foster, G., He, X. & Shimizu, C. The rise and fall of startups: Creation and destruction of revenue and jobs by young companies. Aust. J. Manag. 40 , 6–35 (2015).

Which vaccine saved the most lives in 2021?: Covid-19. The Economist (Online) (2022). noteName - AstraZeneca; Pfizer Inc; BioNTech SE; Copyright - Copyright The Economist Newspaper NA, Inc. Jul 14, 2022; Last updated - 2022-11-29.

Oltermann, P. Pfizer/biontech tax windfall brings mainz an early christmas present (2021). noteName - Pfizer Inc; BioNTech SE; Copyright - Copyright Guardian News & Media Limited Dec 27, 2021; Last updated - 2021-12-28.

Grant, K. A., Croteau, M. & Aziz, O. The survival rate of startups funded by angel investors. I-INC WHITE PAPER SER.: MAR 2019 , 1–21 (2019).

Google Scholar  

Top 20 reasons start-ups fail - cb insights version (2019). noteCopyright - Copyright Newstex Oct 21, 2019; Last updated - 2022-10-25.

Hochberg, Y. V., Ljungqvist, A. & Lu, Y. Whom you know matters: Venture capital networks and investment performance. J. Financ. 62 , 251–301 (2007).

Fracassi, C., Garmaise, M. J., Kogan, S. & Natividad, G. Business microloans for us subprime borrowers. J. Financ. Quantitative Ana. 51 , 55–83 (2016).

Davila, A., Foster, G. & Gupta, M. Venture capital financing and the growth of startup firms. J. Bus. Ventur. 18 , 689–708 (2003).

Nann, S. et al. Comparing the structure of virtual entrepreneur networks with business effectiveness. Proc. Soc. Behav. Sci. 2 , 6483–6496 (2010).

Guzman, J. & Stern, S. Where is silicon valley?. Science 347 , 606–609 (2015).

Article   ADS   CAS   PubMed   Google Scholar  

Aldrich, H. E. & Wiedenmayer, G. From traits to rates: An ecological perspective on organizational foundings. 61–97 (2019).

Gartner, W. B. Who is an entrepreneur? is the wrong question. Am. J. Small Bus. 12 , 11–32 (1988).

Thornton, P. H. The sociology of entrepreneurship. Ann. Rev. Sociol. 25 , 19–46 (1999).

Eikelboom, M. E., Gelderman, C. & Semeijn, J. Sustainable innovation in public procurement: The decisive role of the individual. J. Public Procure. 18 , 190–201 (2018).

Kerr, S. P. et al. Personality traits of entrepreneurs: A review of recent literature. Found. Trends Entrep. 14 , 279–356 (2018).

Hamilton, B. H., Papageorge, N. W. & Pande, N. The right stuff? Personality and entrepreneurship. Quant. Econ. 10 , 643–691 (2019).

Salmony, F. U. & Kanbach, D. K. Personality trait differences across types of entrepreneurs: A systematic literature review. RMS 16 , 713–749 (2022).

Freiberg, B. & Matz, S. C. Founder personality and entrepreneurial outcomes: A large-scale field study of technology startups. Proc. Natl. Acad. Sci. 120 , e2215829120 (2023).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Kern, M. L., McCarthy, P. X., Chakrabarty, D. & Rizoiu, M.-A. Social media-predicted personality traits and values can help match people to their ideal jobs. Proc. Natl. Acad. Sci. 116 , 26459–26464 (2019).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Dalle, J.-M., Den Besten, M. & Menon, C. Using crunchbase for economic and managerial research. (2017).

Block, J. & Sandner, P. What is the effect of the financial crisis on venture capital financing? Empirical evidence from us internet start-ups. Ventur. Cap. 11 , 295–309 (2009).

Antretter, T., Blohm, I. & Grichnik, D. Predicting startup survival from digital traces: Towards a procedure for early stage investors (2018).

Dworak, D. Analysis of founder background as a predictor for start-up success in achieving successive fundraising rounds. (2022).

Hsu, D. H. Venture capitalists and cooperative start-up commercialization strategy. Manage. Sci. 52 , 204–219 (2006).

Blank, S. Why the lean start-up changes everything (2018).

Kaplan, S. N. & Lerner, J. It ain’t broke: The past, present, and future of venture capital. J. Appl. Corp. Financ. 22 , 36–47 (2010).

Hallen, B. L. & Eisenhardt, K. M. Catalyzing strategies and efficient tie formation: How entrepreneurial firms obtain investment ties. Acad. Manag. J. 55 , 35–70 (2012).

Gompers, P. A. & Lerner, J. The Venture Capital Cycle (MIT Press, 2004).

Shane, S. & Venkataraman, S. The promise of entrepreneurship as a field of research. Acad. Manag. Rev. 25 , 217–226 (2000).

Zahra, S. A. & Wright, M. Understanding the social role of entrepreneurship. J. Manage. Stud. 53 , 610–629 (2016).

Bonaventura, M. et al. Predicting success in the worldwide start-up network. Sci. Rep. 10 , 1–6 (2020).

Schwartz, H. A. et al. Personality, gender, and age in the language of social media: The open-vocabulary approach. PLoS ONE 8 , e73791 (2013).

Plank, B. & Hovy, D. Personality traits on twitter-or-how to get 1,500 personality tests in a week. In Proceedings of the 6th workshop on computational approaches to subjectivity, sentiment and social media analysis , pp 92–98 (2015).

Arnoux, P.-H. et al. 25 tweets to know you: A new model to predict personality with social media. In booktitleEleventh international AAAI conference on web and social media (2017).

Roberts, B. W., Kuncel, N. R., Shiner, R., Caspi, A. & Goldberg, L. R. The power of personality: The comparative validity of personality traits, socioeconomic status, and cognitive ability for predicting important life outcomes. Perspect. Psychol. Sci. 2 , 313–345 (2007).

Article   PubMed   PubMed Central   Google Scholar  

Youyou, W., Kosinski, M. & Stillwell, D. Computer-based personality judgments are more accurate than those made by humans. Proc. Natl. Acad. Sci. 112 , 1036–1040 (2015).

Soldz, S. & Vaillant, G. E. The big five personality traits and the life course: A 45-year longitudinal study. J. Res. Pers. 33 , 208–232 (1999).

Damian, R. I., Spengler, M., Sutu, A. & Roberts, B. W. Sixteen going on sixty-six: A longitudinal study of personality stability and change across 50 years. J. Pers. Soc. Psychol. 117 , 674 (2019).

Article   PubMed   Google Scholar  

Rantanen, J., Metsäpelto, R.-L., Feldt, T., Pulkkinen, L. & Kokko, K. Long-term stability in the big five personality traits in adulthood. Scand. J. Psychol. 48 , 511–518 (2007).

Roberts, B. W., Caspi, A. & Moffitt, T. E. The kids are alright: Growth and stability in personality development from adolescence to adulthood. J. Pers. Soc. Psychol. 81 , 670 (2001).

Article   CAS   PubMed   Google Scholar  

Cobb-Clark, D. A. & Schurer, S. The stability of big-five personality traits. Econ. Lett. 115 , 11–15 (2012).

Graham, P. Do Things that Don’t Scale (Paul Graham, 2013).

McCarthy, P. X., Kern, M. L., Gong, X., Parker, M. & Rizoiu, M.-A. Occupation-personality fit is associated with higher employee engagement and happiness. (2022).

Pratt, A. C. Advertising and creativity, a governance approach: A case study of creative agencies in London. Environ. Plan A 38 , 1883–1899 (2006).

Klotz, A. C., Hmieleski, K. M., Bradley, B. H. & Busenitz, L. W. New venture teams: A review of the literature and roadmap for future research. J. Manag. 40 , 226–255 (2014).

Duggan, M., Ellison, N. B., Lampe, C., Lenhart, A. & Madden, M. Demographics of key social networking platforms. Pew Res. Center 9 (2015).

Fisch, C. & Block, J. H. How does entrepreneurial failure change an entrepreneur’s digital identity? Evidence from twitter data. J. Bus. Ventur. 36 , 106015 (2021).

Brush, C., Edelman, L. F., Manolova, T. & Welter, F. A gendered look at entrepreneurship ecosystems. Small Bus. Econ. 53 , 393–408 (2019).

Kanze, D., Huang, L., Conley, M. A. & Higgins, E. T. We ask men to win and women not to lose: Closing the gender gap in startup funding. Acad. Manag. J. 61 , 586–614 (2018).

Fan, J. S. Startup biases. UC Davis Law Review (2022).

AlShebli, B. K., Rahwan, T. & Woon, W. L. The preeminence of ethnic diversity in scientific collaboration. Nat. Commun. 9 , 1–10 (2018).

Article   CAS   Google Scholar  

Żbikowski, K. & Antosiuk, P. A machine learning, bias-free approach for predicting business success using crunchbase data. Inf. Process. Manag. 58 , 102555 (2021).

Corea, F., Bertinetti, G. & Cervellati, E. M. Hacking the venture industry: An early-stage startups investment framework for data-driven investors. Mach. Learn. Appl. 5 , 100062 (2021).

Chapman, G. & Hottenrott, H. Founder personality and start-up subsidies. Founder Personality and Start-up Subsidies (2021).

Antoncic, B., Bratkovicregar, T., Singh, G. & DeNoble, A. F. The big five personality-entrepreneurship relationship: Evidence from slovenia. J. Small Bus. Manage. 53 , 819–841 (2015).

Download references

Acknowledgements

We thank Gary Brewer from BuiltWith ; Leni Mayo from Influx , Rachel Slattery from TeamSlatts and Daniel Petre from AirTree Ventures for their ongoing generosity and insights about startups, founders and venture investments. We also thank Tim Li from Crunchbase for advice and liaison regarding data on startups and Richard Slatter for advice and referrals in Twitter .

Author information

Authors and affiliations.

The Data Science Institute, University of Technology Sydney, Sydney, NSW, Australia

Paul X. McCarthy

School of Computer Science and Engineering, UNSW Sydney, Sydney, NSW, Australia

Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, Australia

Xian Gong & Marian-Andrei Rizoiu

Oxford Internet Institute, University of Oxford, Oxford, UK

Fabian Braesemann & Fabian Stephany

DWG Datenwissenschaftliche Gesellschaft Berlin, Berlin, Germany

Melbourne Graduate School of Education, The University of Melbourne, Parkville, VIC, Australia

Margaret L. Kern

You can also search for this author in PubMed   Google Scholar

Contributions

All authors designed research; All authors analysed data and undertook investigation; F.B. and F.S. led multi-factor analysis; P.M., X.G. and M.A.R. led the founder/employee prediction; M.L.K. led personality insights; X.G. collected and tabulated the data; X.G., F.B., and F.S. created figures; X.G. created final art, and all authors wrote the paper.

Corresponding author

Correspondence to Fabian Braesemann .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this Article was revised: The Data Availability section in the original version of this Article was incomplete, the link to the GitHub repository was omitted. Full information regarding the corrections made can be found in the correction for this Article.

Supplementary Information

Supplementary information., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

McCarthy, P.X., Gong, X., Braesemann, F. et al. The impact of founder personalities on startup success. Sci Rep 13 , 17200 (2023). https://doi.org/10.1038/s41598-023-41980-y

Download citation

Received : 15 February 2023

Accepted : 04 September 2023

Published : 17 October 2023

DOI : https://doi.org/10.1038/s41598-023-41980-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

• News & Events
• About Samuel DuBois Cook
• Director’s Message
• Distinguished Fellows
• Career Achievement Award
• Outside Opportunities
• Class, Wealth, & Social Mobility
• Health Equity
• Education Equity & Policy
• Visual Narratives of Inequality
• Environmental Justice
• Policing Enforcement & Justice
• Artificial Intelligence
• Journal Articles
• Global Inequality Research Initiative (GIRI) Seminar
• Hank & Billye Suber Aaron Young Scholars Summer Research Institute
• Inequality Studies Minor
• Graduate Programs
• Doctoral Fellows Program
• Postdoctoral Programs
• Diversity Initiative for Tenure Economics (DITE)
• Shame of Chicago
• Durham’s Black Wall Street Exhibit
• Past Newsletters

Associate Director Keisha Bentley-Edwards Publishes Paper on Infectious Disease Transmission in a Small Community

Cook Center Associate Director of Research, Dr. Keisha Bentley-Edwards , has co-authored a research article in PLOS ONE titled, “ Homophily and social mixing in a small community: Implications for infectious disease transmission “.

Dr. Bentley-Edwards and her co-authors explored how people in a small, diverse community interact and what this means for the spread of diseases like COVID-19. They focused on who people spend time with based on age, race, and gender. To do this, they talked to people who recently had COVID-19, asked them about their close contacts, and invited those contacts to join the study too.

They found that people tend to spend more time with others who share their race or ethnicity, more than you would expect based on the overall population makeup. Black or African-American participants, in particular, had interactions that spanned a wide range of ages, often because of family connections. In contrast, non-binary individuals tended to interact with people of similar ages.

This tendency for people to stick within their own groups could influence how diseases spread. Understanding these social patterns can help predict how an epidemic might spread and inform strategies like vaccination campaigns to protect those at higher risk.

Access the full journal article here .