The Effect of Learning Factory Approach on Technology Management Education for Social Science Students

To cultivate technology management talents for industry, higher education institutions have developed technology management education since the 1990s. The purpose of introducing technology management course into non-technology discipline, especially Business School, aims to help enterprises foster talents to make appropriate technology-related decisions. This work addresses students‟ conception performance of learning technology management. This research employs a modified Importance-Performance Analysis method to analyze the course contents‟ understandability before and after learning by a questionnaire survey. As the results show, the mean scores of almost all themes before learning are below average and median. However, after class, the scores show minor deviation and notable progress, almost twice the scores before learning. That is to say; a scenario-based learning factory approach is effective for the nontechnology students to learn technology management knowledge. Besides, there are two teaching themes with low prior knowledge creating major performance and nine teaching units with little prior knowledge creating less improvement. The teaching effort should be addressed in these nine teaching units to improve learning performance in technology management.

manufacturing technology, new product or service development, social and ethical issues, government policies, integration of technology and business strategy in MBA programs. Tas and Yeloglu (2018) proposes interconnected and interdependent management courses such as strategic management, project management, R&D management, new product development, innovation management, entrepreneurship, and information management.
Besides, several research suggest incorporating new disciplines to technology management education. This proposals include technology entrepreneurship to offer opportunities (Mosey, 2016),intellectual property management to support sub-disciplines including entrepreneurship, technology strategy, and technology transfer (Fishman, 2013), computing and communications technology to make more effective the management of technology education (Horwitch& Stohr,2012), and curriculum to support courses like engineering science, general business, technology processes (Hauck, 1999). Moreover, Koral-Kordova and Frank(2014) advocates system-thinking perspective to figure out the interrelations among components as a whole. TasandYeloglu(2017) depicts the complete interconnection network between technology management with business management, knowledge management, strategic management, innovation management, R&D management, and computer information system.
Nevertheless, even though a wide variety of graduate and undergraduate courses and curriculums have been proposed and implemented in technology management program, there is only little literature on the effectiveness of these design up to now. For instance, Downs and McAllen (2012)investigates the learning performance of students in an undergraduate technology management program with several factors. Tas and Yeloglu (2018) identifies seven critical knowledge and skills and thirteen key learning outcomes in an undergraduate technology management program. Therefore, this study would like to addresses undergraduate students' degree of understanding on subjects of technology management courses. This article expects that the results will illustrate the learning performance to reduce the gap between the technology management skills required in the industry and learned at the university.

Literature Review Technology Management Education
The rapid development of technologies was one of the most dramatic changes in the past decades. How to manage technology is becoming more important for increasing competitive advantage within organizations. To cultivate technology management talents, higher education institutions have to take the initiatives to develop courses, curriculums, programs, and even departments to accord with the demands of industries and enterprises. The purpose of introducing technology management course into non-technological background fields, especially Business School, is to help enterprises to foster qualified people to make appropriate technology-related decisions. In general, the discipline of technology management education includes strategic management, project management, R&D management, new product development, innovation management, entrepreneurship, and information management (Tas& Yeloglu, 2018). Because management of technology received increasing attention in the 2000s, various universities offer technology management related programs in Canada and US (Yanez&Walsh, 2010;Brent& Pretorius, 2008;Zehner, 2000). Although numerous graduate programs on technology management have been established in universities around the world since then, there is still a lack of education curriculum for technology management at the undergraduate level in universities. That means academic institutions pay less attention to the demands of undergraduate programs on technology management education. To offer a wide-ranging management of technology in undergraduate programs, the demands of industry need to be re-evaluated. The fact is that industries need not only engineers and technical professionals but also nonscience graduates with technology management skill to generate innovative ideas and solutions employing business management related knowledge (Tas&Yeloglu, 2017).

Learning Factory
Since a consortium established by Penn State University was awarded by US National Science Foundation to develop a learning factory in 1994, the concept of "learning factory" has been gradually prevailed in the past years (Jorgensen,Lamancusa,Zayas-Castro& Ratner, 1995). The application of learning factory in academic scenario can be represented by TU Wien Learning and Innovation Factory in Vienna. It is established for integrative production education to provide a physical educational platform for activity-based courses to offer real experiences for students to have a better understanding of the integrative product development process (Sihn&Jager2012). This integration includes steps like product planning and design, engineering, manufacturing, assembly as well as quality assurance. Because these hands-on trainings always comprise exercises to analyze, propose, construct and optimize products or process, different teaching methods are usually combined to use in a learning team (Jager,Mayrhofer, Kuhlang, Matyas& Sihn, 2013). In this sense, problem-based learning factory accentuates that learning involves the construction of knowledge. This could be accomplished through students" passionate efforts to blend new information to their well-known knowledge to materialize their own distinctive experiences (Montero&González, 2009). In this case, the students have to work with team members sincerely and each of them has to play appropriate role to collaborate with each other to achieve their goals (Kolmos& Graaff, 2014;Kolmos, Krogh, & Fink, 2004;Askehave, Linnemann Prehn, Pedersen&Thorso Pedersen, 2015). Because students" studies in groups to learn from one another by solving problems can enhance learning (Gbamanja, 1989).It is an appropriate way for non-science students to learn technology management through the concept of learning factory (Mahmud,Tamyez& Nor, 2019).

Learning Performance
As regards the evaluation of understanding subject, learning performance measurement is a necessary approach. In addition to direct measures of scores of examination, paper, etc., indirect measures of students" perception of their own learning achievement provide an opportunity for students to examine their own learning engagement. No matter direct or indirect measure, learning performance could be measured by comparing scores or perceptions before and after learning. Student"s level of academic achievement is usually considered as the principal outcome of learning processes (Mccrindle& Christensen,1995).The primary and fundamental learning outcome assessment of student usually applies direct measures of student performance such as scores of examination, reports, presentations, etc. On the other hand, indirect measures of student performance are also used to evaluate students" perceptions of their own learning accomplishment. These kinds of measures offer opportunities for students to contemplate their learning process and experience. The self-reporting result may reflect what students believe they have learned form course teaching (Office of Academic Planning & Assessment, 2017). Even though self-reported student learning outcomes are not assumed equal to direct measures of student performance, self-reported learning performances provide valuable learning-related information from student perspectives for improving teaching practices in accordance with important learning objectives (Anaya, 1999).Various higher education studies propose that the self-reported cognitive outcomes of indirect measures have correlation with direct assessments (Pike,1995). Gonyea (2005) proposes that incorporating student selfreported method with other approaches can grant a general measure of student achievement.
Weinstein andMayer (1986) marked a distinction between learning outcome and learning performance evaluated by assessment techniques. The argument proposes that learning outcome stands for the knowledge structures been formed because of learning process. On the contrary, student performance is prone to be affected by these knowledge structures. Furthermore, the nature of the new knowledge structures will influence considerably the future learning although performance on assessment instruments is apparently influential (Trigwell&Prosser, 1991).Santikarn andWichadee (2017) explored students" perceptions on the learning performance of flipped classroom. After compare surveys of autonomy perceptions before and after the intervention (flipped classroom approach), students" average perception scores of all measured performances increase to high levels. That means the learning approach of flipping classroom has an influence on students" learning performance. Furthermore, many research results of pre-tests and post-tests show students" learning improvement, mastery of the subject, and understanding of the content and high achievement (Enfield, 2013;Kong, 2014;Talley& Scherer,2013).

Importance-Performance Analysis
Since Importance-Performance Analysis (IPA) was developed to help to understand customer satisfaction and determine priority of automobile dealers" services (Martilla & James, 1997),it has been widely used as an enormously prevalent managerial tool in diverse fields for performance evaluation. As a graphical method, IPA originally employs a two-dimensional coordinate system by using performance as horizontal axis and importance as vertical axis. Performance denotes the user's perception of the service quality provided by the organization whereas importance stands for the users" assessment of the importance of these services (Yildiz, 2011). The scores of perceived importance and performance of tested targets therefore form point coordinates to be placed on an importance-performance grid plane. Besides, the horizontal axis and vertical axis will be separated into two sides, the low and the high, to establish a four-quadrant system. The IPA chart is illustrated as Fig. 1. The separation line of the four quadrants could be the middle of the scale of x-and y-axis or the average values of importance and performance. Source: Martilla and James (1977) As shown in Fig. 1, when customers consider a service highly important but perform poorly, this service will fall into Quadrant I (high importance/low performance) and requires immediate remedial action so that this is called "concentrate here" quadrant. Quadrant II (high importance/high performance) includes services which are highly important perceived by customers and is performed very well by service providers. These elements represent the competitive advantage of providers so that deserve continuing to maintain the quality of these services so as to be referred to "keep up with the good work" quadrant. If the elements are located in quadrant III (low importance/low performance), it means customers do not perceive these services to be very important and to be well performed. Therefore, these elements do not deserve allocating resources to take prompt action to improve these services. This quadrant is called "low priority". Quadrant IV (low importance/high performance) is called "possible overkill" since it includes elements been perceived as not so important while perform so well. Because these high quality services cannot increase customer satisfaction, the excessive resources allocated on these services should be reassessed (Levenburg&Magal, 2004). That is to say, it is better to reallocate these overinvested resources to other services that will increase customer satisfaction to fulfill the maximum benefit of the organization.
The original method is later called scale centered quadrant model (SCQM) owing to the cross section of coordinates being in the middle of the scale in the original graph (Martilla & James, 1977). However, the fact that most attributes with average values of importance and performance would fall into Quadrant I (keep up the good work) by SCQM results in a weak discriminative power of the IPA graph(A. Rial, J.Rial, Varela, & Real, 2008). To improve this defect, Data centered quadrant model (DCQM) using the mean or median values of importance and performance as the cross section of coordinates was proposed. In fact, Martilla andJames (1977) states that how to position the importance and performance axes on the grid is a matter of judgment and median is supposedly preferable to mean as a measure of central tendency because there may be no true interval scale. Obviously, the biggest flaw of the above two QM methods is that even minor element shifts in the coordinate system can produce radical changes in the identification of priorities for improvement (Bacon, 2003;Tontini, Picolo&Silveira, 2014).
According to the abovementioned locating of elements to quadrant, the top and low priorities for improvement of the services could be decided. In the past decades, numerous researchers revised this graph method to meet the mathematical quadrant system because the quadrant arrangement of the original IPA graph numbering clockwise from the upper left (northwest) quadrant is not consistent with standard mathematic expression in which the numbering of a two-dimensional Cartesian system moves counter-clockwise starting from the upper right (northeast) quadrant (Ormanović, Ćirić1, Talović, Alić, Jelešković&Ĉaušević, 2017; Deepa&Rupashree, 2019;Sum, Champahom, Jomnonkwao&Ratanavaraha, 2019).
As the above-mentioned discussions show, the learning performance of Technology Management education by scenario-based learning factory approach deserves in-depth study. Besides, exactly because IPA is a mature method to compare the gap between importance and performance of consumers" perception, this research modifies IPA method to compare students" conception performance before learning and after learning. To conduct a survey to evaluate students" learning performance, this research designs a self-report questionnaire to elicit students' perceived understandability of each teaching unit theme of technology management course by IPA.

Research Methodology Research Design
Because student"s self-report of learning performance provides a reliable estimate of actual academic achievement (Barnett&Hixon, 1997;Klein, 1998;Purdie&Hattie, 2002). Various research employ students" conceptions of course evaluation items to assess learning performance. These research analyze the difference between pre-learning and post-learning (also called pre-teaching vs. post-teaching, pre-lecture vs. post-lecture, pre-test vs. post-test) responses to the same questionnaire items (Chotimah&Rafi,2018;Hayes,Holden, Gaynor, Kavanagh, & Otoom, 2013;Fleming-Castaldy, 2018).
According to this approach, this study utilizes survey method to collect the self-reported perception data of students taking the course of "Management of Technology" at Department of international Business in National Taichung University. This course intends to enhance students" ability of understanding, communicating and applying technology management knowledge. The course design is based on problem-based learning approach. At first, all students accept in-class lecture and discussion on technology management for ten weeks. In the second stage, they are required to decide their own topic to conduct a case study on company, product, or service by applying their learned knowledge and by using a scenario-based learning factory approach.

Instruments
To conduct a survey to assess student"s conception of learning, this study uses questionnaire as research instrument. The self-administered questionnaire containing twenty questions is based upon previous research (Tas&Yeloglu, 2018;Downs &McAllen, 2012) as well as the teaching units and curriculum design of Technology Management of course. It is used to conduct pairwise comparison of understandability of teaching theme before and after learning (denoted as pre-learning and post-learning).
Students were asked to rate their perceived performance of each item according to a 0 to 10 scale with the meaning from completely unknown to completely understanding. The questionnaire is designed as TABLE I.

Respondents
All students were asked to participate voluntarily in the teaching experiment by filling the questionnaire before and after in-class teaching and case study to reveal their perceived understandability of teaching unit themes. No identifying information was collected and no incentives were offered to motivate students" participation. The questionnaires were delivered to 46 students taking the course of "Management of Technology" in fall semester 2019. The participants were sophomores at the Department of International Business in College of Management of National Taichung University of Education. Forty-six copies of questionnaire were collected with 100% response rate. After screening invalid response, a total of 43 records were valid with 93.48% effective response.

Analysis Procedure
This study included six stages of data and IPA analysis by using IBM SPSS Statistics 20 as follows. i.
Verify the validity and reliability of the questionnaire.
ii. Average the self-evaluated scores before and after learning of all respondents for each question.
iii. Calculate the differences between the average scores of post-and pre-learning of each question to be improvement.
iv. Assign pre-learning (post-learning) scores as x-axis (y-axis) to get coordinates (x, y) to draw a scatter plot.
v. Calculate the average pre-learning (post-learning) score of all questions and assign it as the cutting point of quadrants on x-axis (y-axis); then use this pair of (x, y) as the reference point of four quadrants.
vi. Compare each pair of pre-post learning score of the 20 questions with the reference point and arrange them to the four quadrants.

Results
The goal of construct validity test is to verify how well a set of measured items in questionnaire. This paper assessed internal construct validity to inspect the corrected item-total correlation for overlap by using correlation coefficient = 0.4 as the cutting point for adequate correlation (Chotimah& Rafi, 2018;Hayes,Holden, Gaynor, Kavanagh&Otoom, 2013).
The corrected item-total correlation scores of pre-learning instrument varied in the range of 0.498 to 0.887 and post learning instrument with scores ranging from 0.570 to 0.888. The item-total correlations corrected for overlap were >0.4 for all items indicating good relationship. Besides, all the test items of pre-learning and post-learning were significant at the 0.01 level. It means that the 20 items are valid, so the research can use the measures in assessing the students" scores of pre-learning and post-learning without any reconstruction.
As for the reliability of questionnaire, various reliability tests were proposed to assess the quality of the measurement to collect data. This research used the internal consistency reliability test by analyzing the Cronbach"s Alpha values to check the internal consistency of the pre-learning and post-learning measures (Chotimah& Rafi, 2018;Chin, Choi, Chan& Wong;Tapsir, Pa& Zamri,2018). As a result, the Cronbach"s Alpha values for pre-learning and post-learning are 0.959 and 0.973 respectively. The high reliability means that these questions will measure the same concepts consistently and can be used for the following analysis. Moreover, deleting any item will not increase reliability. The results are shown as TABLE Ⅱ.

TABLEⅡ. STATISTICS OF VALIDITY AND RELIABILITY.
After calculate the self-report post-and pre-learning as procedure 2 & 3, the results in TABLE III shows that the range of mean of before class is 2.25, which is far higher than that of after class (0.81). Combing the fact that mean of after class (8.25) is far greater than that of before class (4.41), that means the learning performances of the 20 items increase significantly and reach almost to the same consistent level despite of the various prior knowledge level. For scores before learning, the mean score of Item 17 "innovation leader might fail" ranks as the highest (5.51) while Item 2 "the difference between high, medium and low technology" ranks as the lowest (3.26). For scores after learning, the mean score of Item 17 "innovation leader might fail" still ranks as the highest (8.67) while Item 6 "technology will cause long term circulation of economics" ranks as the lowest (7.86). As for the improvement, the mean score of Item 2 "the difference between high, medium and low technology" ranks as the highest (4.84) while Item 14 "technology leader could define industry standards" ranks as the lowest (3.09). Based on theTABLE III., IPA analysis denoted scores of post-and pre-learning as x-axis and y-axis correspondingly. After drawing a scatter plot by using the pair of mean scores before and after class, the average of mean scores before and after teaching (4.41, 88.25) were denoted as the reference point. Afterwards, the 20 items were assigned to the four quadrants. Items of 4,5,9,13,14,15,16,17 and 19 belonged to quadrant I, items 8 and 12 to quadrant II, and items of 1, 2, 3, 6, 7, 10, 11, 18and20 to quadrant III. The results are plotted as Fig. 2. As for the fact that no item falls into quadrant IV, it means that there is no item with high prior knowledge and minor improvement. On the contrary, two items (8 and 12) falling in quadrant II means low prior knowledge items have major performance. As for the nine items in quadrant III, little prior knowledge with less improvement means teaching effort should be address on.

Conclusions
This study has identified the themes critical to technology management course and learning performance of these themes. According to the analysis result, the mean scores of almost all themes before class are below average and median. This might result from the fact that the respondents were all non-science background students. However, the scores after class showed not only minor deviation but also notable progresses almost twice the scores before teaching. That is to say, the teaching method of case study by learning factory concept is effective for the non-science students to learn the course of technology management. As for the result of quadrant II in modified IPA analysis, students knew a little about the concepts of "the discovery of science might never been commercialized" and "technology leader will has a higher reputation" before class, but gained a plenty of knowledge of them after teaching. It implies that it is possible to bridge the knowledge gap about marketability of science and technology by providing technology management course for business school students. Furthermore, findings indicate that the most improvement theme is "the difference between high, medium and low technology" and the least improvement theme is "technology leader could define industry standards" falling in quadrant III and I respectively not in quadrant II theoretically. It suggests that the modified IPA cannot completely explain the margin of progress and is necessary to complement with other methods. Finally, this study suggests that future studies can employ two groups of science and non-science students to evaluate and compare their learning performance in technology management.