Article

Evaluation of the Effectiveness of a University-Society Engagement Project in the Field of Mechanics

Evaluación de la Eficacia de un Proyecto de Vinculación con la Sociedad en el Área de Mecánica

Víctor Lenin Montaño Roldan[*]

Luis Hidalgo Solorzano*

Martin Mateo Ramírez Márquez*

Jerson Valdez Ibarra*

Abstract

The evaluation of university-society engagement projects remains an unresolved challenge in Latin American higher education, and Ecuador is no exception. This research set out to do just that: go beyond the numbers to understand the true impact of the “Strengthening Technical Skills for Community Development” project, implemented by the Mechanical Engineering program at the School of Engineering of the Luis Vargas Torres Technical University of Esmeraldas (UTLVTE) during the first semester of 2025. The initiative consisted of a technical training program in arc welding, in which 52 young people from Esmeraldas—from vulnerable communities (the Codesa sector)—participated. In working with them, the university not only shared knowledge but also demonstrated a willingness to listen, trust, and commitment. Using a mixed-methods approach, quantitative tools were combined with an in-depth qualitative analysis of the participants’ perceptions. The results were compelling: 100% of the beneficiaries reported being fully satisfied with the technical content they received, the facilitators’ performance during the learning process, and the practical applicability of what they learned in their daily lives. Likewise, 80.76% rated the materials provided positively , and the project achieved an overall efficiency of 90.47%, reflecting committed and well-directed management. During the process, a profound transformation was observed in the participants’ self-esteem and outlook on life. These findings were compared with international methodological frameworks and the faculty’s prior experiences, which allowed for the validation of the process’s coherence and paved the way for future replications. The project demonstrated high effectiveness in its technical, social, and educational dimensions, establishing itself as a replicable model for new community engagement initiatives in the field of engineering.

Keywords: Project evaluation, community engagement, educational effectiveness, welding, technical training, impact indicators.

Resumen

La evaluación de los proyectos de vinculación entre la universidad y la sociedad sigue siendo un desafío pendiente en la educación superior latinoamericana, y Ecuador no es la excepción. Esta investigación se propuso justamente eso: ir más allá de los números para comprender el verdadero impacto del Proyecto “Fortalecimiento de Competencias Técnicas para el Desarrollo Comunitario”, ejecutado por la Carrera de Mecánica de la Facultad de Ingenierías de la Universidad Técnica Luis Vargas Torres de Esmeraldas (UTLVTE) durante el primer semestre de 2025. La iniciativa consistió en un proceso de capacitación técnica en soldadura por arco voltaico, del que participaron 52 jóvenes esmeraldeños provenientes de sectores vulnerables (sector Codesa). Frente a ellos, la universidad no solo desplegó conocimientos, sino también escucha, confianza y compromiso. A través de un enfoque metodológico mixto, se combinaron herramientas cuantitativas, con un análisis cualitativo profundo de las percepciones de los participantes. Los resultados obtenidos fueron contundentes: el 100% de los beneficiarios manifestó estar plenamente satisfecho con el contenido técnico recibido, el desempeño de los facilitadores durante el proceso de aprendizaje y la aplicabilidad práctica de lo aprendido en su vida cotidiana. Asimismo, el 80,76% valoró positivamente los recursos materiales entregados, y el proyecto alcanzó una eficiencia global del 90,47%, lo que refleja una gestión comprometida y bien orientada. Durante el proceso, se observó una transformación profunda en la autoestima y en la proyección de vida de los participantes. Estos hallazgos fueron contrastados con marcos metodológicos internacionales y con experiencias previas del cuerpo docente, lo que permitió validar la coherencia del proceso y abrir caminos para futuras réplicas. El proyecto demostró una alta efectividad en sus dimensiones técnica, social y educativa, consolidándose como un modelo reproducible para nuevas intervenciones de vinculación en el campo de la ingeniería.

Palabras clave: Evaluación de proyectos, vinculación con la sociedad, eficacia formativa, soldadura, formación técnica, indicadores de impacto.

Introduction

Evaluating the effectiveness of university-community engagement projects is a theoretical and methodological challenge for higher education today. As Wanjiru and Liu point out (2022) , “university-community engagement can take many forms and impact multiple stakeholders, and an evaluation focused solely on university perspectives might overlook the community perspective, which is equally important” (p. 45) . This complexity is heightened in contexts of high social vulnerability, where technical training projects seek not only to transfer knowledge but also to bring about profound changes in the lives of participants.

Various studies agree on a fundamental point: to truly understand what happens in these processes, we need more robust methodologies capable of capturing their complexity without oversimplifying it. Bekele et al. (2024) note that “none of the existing methodological and conceptual frameworks provides a holistic analysis of meaningful engagement indicators that transcend specific contexts of application,” which makes “the collection of data on university-society engagement and the development of meaningful interpretations a challenge for researchers and professionals” (p. 202) . In the Ecuadorian context, this concern is shared by Jaramillo Tituaña et al. (2025) , who note that “the lack of a clear and standardized methodology for evaluating engagement projects can result in a subjective and inconsistent assessment of the results obtained, which hinders informed decision-making and the continuous improvement of programs” (p. 317) .

In response to this need, proposals have emerged, such as that of Gendle and Tapler (2024) , who developed the Community Engagement and Partnership Inventory (CEPI), a tool that “provides a systematic evaluation of programs, applying the principles of critical global inquiry in such a way that the administration and scoring methods can be flexible and adapted to the needs of the program and the institution” (p. 12) . Welding is a trade learned through hands-on practice and a keen eye for detail; therefore, assessing whether a training program is truly effective goes beyond exams. Yotee et al. (2025) demonstrated that “the visual inspection training program using the SCHEMA teaching model for industrial technicians resulted in an 81.52% increase in efficiency” (p. 72) , validating the importance of structured pedagogical models in this field.

“Engagement is a function that allows universities to realign their objectives and visions for the future, while remaining grounded and recognizing themselves as an integral part of society” (Del Cioppo & Bello, 2018, p. 106) . In this regard, Cabrera-Jara et al. (2023) note that “instructional planning is a key element for students to learn how to learn, but for this to occur, there must be constant innovation in teaching and learning methods, as well as planning for the processes involved in teaching and putting what has been learned into practice” (p. 261) . They also clarify that, “today’s universities have incorporated community engagement into their curricula. This is a prerequisite for earning a degree, so that future professionals can acquire knowledge and engage with their field of work according to their specialty” (p. 259) .

The case of Esmeraldas, a province in Ecuador characterized by high levels of structural poverty and youth violence, illustrates the urgency of this type of intervention. The needs assessment for this project revealed that in the target community (Codesa sector), 65% of the population has only a secondary education, which limits access to specialized job skills and perpetuates socioeconomic disparities. This information, gathered through a participatory process, confirmed the importance of welding training, a trade in high demand in local manufacturing industries (agriculture, construction, and metallurgy).

Higher education institutions (HEIs), and in particular the “Luis Vargas Torres” Technical University of Esmeraldas (UTLVTE), have an obligation to address these realities through their core functions. The Academic Regulations of the Council for Higher Education (CES) define engagement with society in Article 4 as the function that “generates capacities and the exchange of knowledge in accordance with the academic fields of the HEIs to ensure the development of effective responses to the needs and challenges of their environment” (Council for Higher Education [CES], 2019) . At the institutional level, Article 7 of the University Statute establishes as one of its objectives “to develop proposals and approaches to address the problems of the province, region, and country” (UTLVTE, 2024) and the Community Engagement Regulations, in Article 2, prioritize “ongoing attention to the needs of the social environment, especially those of vulnerable sectors of the population and Historically Excluded Groups (GHE)” (UTLVTE, 2024) .

This tradition of community engagement has been previously documented within the faculty itself. Viscaino Valencia et al. (2021) evaluated the performance of mechanical engineering students in a knowledge transfer project in the Camarones parish, demonstrating that “all students obtained a score of 8.0 or higher, leading to the conclusion that their knowledge and skills in relation to their professional profile are optimal” (p. 10) . This background demonstrates that the School of Engineering has accumulated experience in this type of initiative and underscores the need to now evaluate not only student performance but also the project’s overall effectiveness in the community.

Therefore, the objective of this study is to evaluate the effectiveness of the community engagement project “Strengthening Technical Skills for Community Development” during its first implementation period in its technical, social, and educational dimensions, proposing an integrative framework that contributes to the development of evaluation methodologies for future interventions in the field of mechanical engineering.

Materials and methods

The research adopted a mixed-methods design, combining the systematization of documentary information on the project with the analysis of quantitative and qualitative data collected during its implementation.

The target population was determined based on a participatory needs assessment conducted in March 2025, which identified the Tabiazo Parish and the Codesa sector (Youth Center) as priority intervention areas. The final sample consisted of 52 young people (out of a target population of 70), selected through simple random sampling, aged between 18 and 35, with an educational profile of 65% high school, 27% technical school, and 8% college. The sample is representativ ly with a 95% confidence level and a sampling error of 5.8%, which is considered acceptable.

The “Strengthening Technical Skills for Community Development” program, implemented during the IS-2025 academic period (September through December 2025), was structured around three components:

Outreach and Assessment: Outreach campaign and administration of needs assessments.

Theoretical and Practical Training: Training in electric arc welding, with an emphasis on safety standards, welding techniques, and supervised practice.

Evaluation and Closure: Administration of satisfaction surveys, statistical analysis, preparation of reports and a scientific article, and the manufacture of 50 chairs for the community.

The methodological design was inspired by previous experiences evaluating community engagement projects within the faculty itself. Viscaino Valencia et al. (2021) employed a mixed-methods approach to evaluate student performance, using a questionnaire that measured three components—knowledge, skills, and attitudes—with a weighting of

T=0.50(K)+0.30(S)+0.20(A). Following this precedent, the present study adapted a similar approach, but focused on measuring the beneficiaries’ perceptions and the project’s efficiency, rather than student performance.

To develop the indicators, we followed the recommendations of Del Cioppo and Bello (2018) , who state that “impact indicators are developed in such a way as to involve the learning process of community-based projects with the participants who evaluate them” (p. 113) . An impact assessment survey using a 5-point Likert scale was administered, evaluating five dimensions: technical content, instructor, resources and materials, practical applicability, and overall organization. Additionally, management indicators were calculated: coverage, activity completion, and goal achievement, resulting in an overall efficiency of 90.47%. Complementarily, participant observation was conducted during the practical sessions, along with an analysis of monthly and semiannual reports.

The questionnaire was subjected to a validation process to ensure its reliability and relevance, following the example of Jaramillo Tituaña et al. (2025) , who reported that their instrument achieved “an average Content Validity Ratio (CVR) of 0.85” (p. 323) .

Quantitative data were processed using descriptive statistics (frequencies and percentages). Qualitative data were analyzed using thematic content analysis, identifying emerging categories related to empowerment.

Results

The impact survey administered to the 52 participants yielded the following results:

Table 1. Level of Satisfaction by Evaluated Dimension

Dimension	Very satisfied	Satisfied	Neutral/Dissatisfied
Technical Content	100%	0%	0%
Instructor	100%	0%	0%
Resources and Materials	80.76%	19.24%	0%
Practical Applicability	100%	0%	0%
General Organization	100%	0%	0%

Source: UTLVTE (2025). APPENDIX 10 - Project Impact: Evaluation of the alignment of the outreach project with the curriculum and learning outcomes.

The results show an optimal and unanimous level of satisfaction across most dimensions. The high level of satisfaction with resources and materials (80.76% “Very satisfied”) confirms the project’s adequate planning and execution.

The analysis of management indicators showed:

Coverage: 71.43% (52 out of 70 target youth)

Activity completion: 100% (18 out of 18 scheduled activities)

Achievement of objectives: 100% (7 out of 7 planned objectives)

Overall Efficiency: 90.47%

Three main categories emerged from the analysis of reports and participant observation:

Self-efficacy and tangible achievement: The production of 50 pieces of office furniture (chairs) as the final product was a key milestone, allowing the young people to see the fruits of their labor. As one participant noted: “I never thought I could make something like this with my own hands.”

Life Outlook and Entrepreneurship: The midterm report concludes that the young people “are capable of applying solid welding skills in their work or academic contexts, which will increase their productivity and employment opportunities,” noting “a high level of empowerment, demonstrated by unanimous satisfaction and greater confidence to launch their own projects.”

Social Recognition: The donation of furniture to the community solidified the Youth Center’s role as an exceptional partner in fostering university-community ties.”

Discussion

The results of this research demonstrate that the evaluated project constitutes a successful model of social intervention through mechanical engineering. The combination of a participatory assessment, high-quality technical training, and a community-based final product generated a multidimensional impact that transcends the mere transfer of knowledge, aligning with the methodological frameworks proposed by the specialized literature.

The finding that 100% of participants were satisfied in key areas such as technical content, instructor performance, and practical applicability validates the program’s relevance and the effectiveness of the methodology used. This result is consistent with previous welding training experiences that highlight the importance of combining theory and practice with a rigorous focus on industrial safety (Hempel & Guidek, 2024) . Likewise, the high level of satisfaction with resources and materials (80.76% “Very satisfied”) confirms the project’s adequate planning and execution, in line with the quality management requirements of the sector (Cabrera Jaramillo & Corozo Arroyo, 2024) .

A comparison of the results of this study with those obtained by Viscaino Valencia et al. (2021) in the Camarones parish reveals a consistent pattern of success in the community engagement projects of the UTLVTE School of Engineering. While the Camarones study conducted by Viscaino et al. (2021) focused on evaluating student performance during knowledge transfer in metalworking processes—documenting that 92% of participants achieved “very satisfactory” levels (p. 9) —the present study complements this perspective by assessing the beneficiaries’ perceptions and the project’s efficiency. Taken together, both studies demonstrate that the faculty has succeeded in establishing a partnership model that is effective both for the education of its students (as shown by Camarones’s study) and for the empowerment of communities (as shown by the present study). This two-way impact is precisely what the Academic Regulations define in Article 4 as a high-quality community engagement initiative: “it builds capacities and fosters the exchange of knowledge in accordance with the academic fields of higher education institutions” (Council for Higher Education [CES], 2019)

The experience in Esmeraldas is part of the contemporary discussion on how to evaluate university engagement. As Bekele et al. (2024) point out, “collecting data on the university’s engagement with society and drawing meaningful interpretations from it is a challenge for researchers and professionals” (p.202) . In this regard, Jaramillo Tituaña et al. (2025) emphasize that “engagement with society must be a two-way street. Higher education institutions must not only transfer knowledge but also allow themselves to be influenced by society’s knowledge and demands, enabling these to impact their curricula, teaching methodologies, and lines of research” (p. 330) . This call for bidirectionality resonates with the findings of our study, in which interaction with the community not only benefited the 52 young people who received training but also enriched the education of the participating college students and generated knowledge applicable to future interventions.

Our study addresses this challenge through a multi-method approach that combines quantitative efficiency indicators (coverage: 71.43%; compliance: 100%) with qualitative evidence of social transformation, following the IAIA recommendations cited by Del Cioppo and Bello (2018) , who note that “Impact Assessment can help us design and implement better policies, plans, programs, and projects that will address major challenges such as climate change, biodiversity loss, population growth, urbanization, conflicts over increasingly scarce resources, inequality, and new technological opportunities”(p. 107) . This approach aligns with the proposal by Wanjiru and Liu (2022) , who developed a conceptual framework “consisting of three domains of university-community engagement: purpose, process, and community impacts” (p. 47) .

The most significant finding of this study is the qualitative dimension of empowerment. The fabrication of furniture served as a catalyst that transformed abstract learning into concrete achievement and social recognition. This impact on well-being and personal growth resonates with international research. The CWB Welding Foundation (2026) documents that “hands-on learning does more than teach technical skills. It builds confidence, reduces anxiety, and creates a sense of belonging. For many students, welding programs have been a lifeline, offering purpose and stability during difficult times.” Similarly, in the welding program at Utah State University, instructor Chloe Wilson observes that “welding changes lives here. Students prove to themselves that they can do something difficult, and that confidence carries over into everything else” (Utah State University, 2025) .

The significance of this approach becomes evident when contrasted with programs focused exclusively on job placement. Byrd (2014) concluded that “all participants in welding training experienced anxiety during welding tests,” and that “a training program integrated with virtual reality will reduce anxiety more effectively than a 100% virtual program” (p. 45) . By prioritizing empowerment and community recognition, our project equipped young people with resilience that protects them from frustration, building human and social capital that extends beyond the wait for formal employment.

Study limitations: The main limitation is that the impact evaluation was conducted immediately after the intervention, leaving the medium-term sustainability of the empowerment unknown—a common challenge in evaluating community engagement projects in the Ecuadorian context (Jaramillo Tituaña & Alcívar Cedeño, 2025).

References

Bekele, T. A., Ofoyuru, D. T., & Woldegiorgis, E. T. (2024). Assessing university-society engagements: Towards a methodological framework.49, 2 , 201–221. Innovative Higher Education. doi:https://doi.org/10.1007/s10755-024-09712-8

Byrd, A. P. (2014). Identifying the effects of human factors and training methods on a welding training program. [Doctoral dissertation, Iowa State University].

Cabrera Jaramillo, L. A., & Corozo Arroyo, G. A. (2024). Proposal for a technical guide for training in welding management focused on metal structures.

Cabrera-Jara, C. A., Luy-Navarrete, W. A., & Merino Velásquez, J. (2023). University planning and sustainability of programs and/or projects involving community engagement. *6 , 12

Council for Higher Education [CES]. (2019). Academic Regulations (RPC-SO-08-No. 111-2019).

CWB Welding Foundation. (February 5, 2026). Building skills and resilience: How the Marinucci Family Foundation transformed welding education and student well-being across Canada. Retrieved from https://www.cwbweldingfoundation.org/building-skills-and-resilience

Del Cioppo, J., & Bello, M. (2018). Social impact indicators for evaluating community engagement projects. Económicas CUC.39, 1 , 105–116. doi:http://dx.doi.org/10.17981/econcuc.39.1.2018.07

Gendle, M. H., & Tapler, A. S. (2024). The Community Engagement and Partnership Inventory (CEPI): An aspirational open-source instrument to assess community-based global learning programs.30, 2 . Michigan Journal of Community Service Learning. Retrieved from https://doi.org/10.3998/mjcsl.4909

Hempel, J. F., & Guidek, D. (2024). Training on Hazards in Welding Processes in Secondary Education. National University of Misiones.

Jaramillo Tituaña, B. A., & Alcívar Cedeño, A. K. (2025). Methodology for Evaluating Community Engagement Projects at the Tena Higher Technological Institute.7, 1 , 315–334. PENTACIENCIAS Multidisciplinary Peer-Reviewed Scientific Journal.

Mesias-Simisterra, Á., Aguilar-Mora, M., Gonzalez-Vega, A., & Reina-Tenorio, L. (2026). Youth empowerment through technical training in welding.

Total Soft Bank; Pusan National University; Korea Polytechnics. (2024). A study on the effectiveness of the virtual welding simulator for welding workforce development. KCI.

Utah State University. (May 15, 2025). From sparks to success: Welding program grows at USU Blanding. Utah State University Today. Utah State University Today. Retrieved from https://www.usu.edu/today/story/from-sparks-to-success-welding-program-grows-at-usu-blanding

Viscaino Valencia, P. R., Aguilar Mora, M. L., Mera Mosquera, A. R., & Montaño Roldan, V. L. (2021). Transfer of mechanical engineering knowledge in the Camarones parish, Esmeraldas canton, Ecuador.4, 2 . Journal of Business and Entrepreneurial Studies. doi:https://doi.org/10.37956/jbes.v4i2.84

Wanjiru, I. R., & Liu, X. (2022). Evaluating university–community engagement through a community-based lens: What indicators are suitable?26 , 2 , 43–58. Journal of Higher Education Outreach and Engagement.

Yotee, S., Dhinnabutra, P., Khrueaphue, T., Thammasang, S., & Laoha, C. (2025). Development and evaluation of a visual welding inspection skills training package using the SCHEMA teaching model for industrial technicians. Journal of Technical and Engineering.