Revista Produção e Desenvolvimento, v.5, n.1, e395: 2019

DOI:  10.32358/rpd.2019.v5.395

 

Educational Reward Information Communication API (ERIC API): A Preliminary Study Result

 

 

ABSTRACT

Educational Resource Information Communication (ERIC) API is used to connect two separate systems while keeping both systems working independently without leaking users’ privacy data. This research uses ERIC API to integrate an educational reward system called Trading Card Game with Moodle, a famous open-source learning management system. When students authorize Moodle to dispatch the rewards (i.e., in-game cards) for completing learning activities (e.g., assignments and quizzes) to their account in the Trading Card Game, Moodle will no information about the credentials that they have in the Trading Card Game. This research conducts a pilot study to understand whether or not students are satisfying with having the API to integrate Moodle and Trading Card Game. The results not only show that ERIC API is acceptable for students but also provide researchers and teachers support of evidence to having a reward system into their learning management system.  

KEYWORD: Learning Management System; Secure Communication; Authorization; Trading Card Game; Educational Reward.

 

1.   INTRODUCTION

Learning management systems such as Moodle and Canvas are already widely used in schools. When considering to integrate another independent system or tool into the learning management system, a concern may be raised – while both of the two applications (e.g., systems and tools) have their own credentials and data, how to prevent one application gaining the students’ information stored in another system accidentally or intentionally?

The research team has developed Educational Reward Information Communication API (ERIC API) that enables two systems' interoperability but keep both systems working independently like they were (Chen, Chang, & Chang, 2016; Chen, Zhao, Luo, Chang, Qian, Kuo, & Chang, 2017). This research conducts a pilot study to understand the perceived usability of the API. In the evaluation, the ERIC API is used to integrate Moodle and an educational reward system call Trading Card Game (Chen, Chang, Kuo, & Heh, 2016; Chen, Kuo, Chang, & Heh, 2017; Chen, Kuo, Chang, & Heh, 2019) so Moodle can give students in-game cards as rewards based on their performances on various learning activities.

Section 2 describes the background of learning management systems and ERIC API. The hypotheses of the perceived usability towards ERIC API and the details of this study are introduced in Section 3. Section 4 analyzes the collected data based on the evaluation plan. Several findings from the analysis results are discussed in Section 5. Section 6 concludes this research with a brief summary and possible future works.

2.   BACKGROUND

2.1.   Learning Management Systems

Learning management systems help teachers monitor students' learning outcomes and allow student accessing learning materials online (Jurubescu, 2008). Moodle is one of the popular learning management systems for online learning (Kasim & Khalid, 2016). Many researchers have developed plug-ins for Moodle; for instances, analyzing students’ learning profiles and interactions to provide teachers learner specific information (Charleer, Santos, Klerkx, & Duval, 2014; Graf & Kinshuk, 2013; Kumar et al., 2014); creating remote simulator as virtual laboratory (Torre, Sanchez, Andrade, & Restivo, 2016); and, notifying students with low attendance in order to decrease the drop rate in the distance education (Almeida, Costa, Sousa, Freitas, Canedo, Prettz, Zacarias, & Galdo, 2016).

However, the cost of developing a plug-in from scratch is much higher than porting lightweight web application to the learning management systems. Vozniuk and colleagues (2015) developed three apps with OpenSocial and ActivityStreams for accessing data from learning environments and showing learning analytics results with dashboards. On the other hand, SocIoS API and framework are designed by Karadara and colleagues to provide developers a uniform access mechanism of accessing data from popular social networks (i.e., Dailymotion, Facebook, FlickR, Google+, Instagram, Twitter, and YouTube) (Karadara, Kalogirou, Papaoikonomou, Varvarigou, & Tserpes, 2014).

 Although OpenSocial API and SocIoS API can help to integrate functions that applications and social media sites provide, the integration requires users provide the learning management systems their credentials of another system that provides the features or data. For instance, although a system 'X' uses OpenSocial API that enables it to access the data on Facebook, a user needs to provide the system 'X' his or her credentials on Facebook if he or she wants the system 'X' to access his or her data on Facebook.

OAuth is an open authorization standard that allows users to grant an OAuth client application to access the resource stored in the OAuth server without sharing their credentials (Ferry, Raw, & Curran, 2015). OAuth is one of the popular token-based authentications that allows users to access the applications by logging on their existing accounts such as Facebook, Twitter and Google. As steps 1 and 2 in Figure 1 show, when a user wants to play Candy Crush, it will ask him or her to authorize it to access the user’s data on Facebook. Doesn’t like earlier system X’s case, Candy Crush asks the user to enter his or her Facebook’s username and password “on Facebook” to grant itself particular permissions as step 3 shows. Once Candy Crush is authorized by the user, Candy Crush will be given a token via OAuth for accessing the required user’s data from Facebook (see steps 4 to 5 in Figure 1).

Figure 1. OAuth workflow

In such case, users do not need to register a new account or sign up again for the applications that supports OAuth. It is important to have a similar mechanism to authorize learning management systems to access data from other learning systems and tools that support students, teachers, and administrative staffs developed by researchers.

If a learning management system can have an OAuth server designed and added on the top and other lightweight applications and widgets can have OAuth client function implemented, then the access of students’ profile, learning history, goals and preferences can be done without issue. However, OAuth solution currently does not exist in most of learning management system. Moreover, in many advanced learning technology solutions both of the systems/tools can be data service providers for each other. Under such circumstance, the OAuth solution may not perfectly fit into the situation where more than one application/tool has its own credentials for its users.

 

2.2.   ERIC API

Chen, Chang, and Chang (2016) proposed and developed an Educational Resource Information Communication API (ERIC API). ERIC API is an application program interfaces (APIs) with class libs and PHP plug-in modules that can be applied to any Internet-based systems. Users of a system 'A' will not need to provide their credentials for another system 'B' that provides the data and functions to system 'A' and the two systems can exchange all of the needed data and information while keeping the two systems still running independently and having database access being private. Figure 2 shows how two systems (i.e., Moodle and a trading card game "TCG") are integrated with ERIC API.

Figure 2. ERIC API Architecture

 

When a Moodle user (at left hand side in Figure 2) wants to grant Moodle the permission of accessing his or her data in the Trading Card Game (at right hand side of Figure 2), Moodle is considered as the service requestor (i.e., the ERIC API client) and the Trading Card Game is the service provider (i.e., ERIC API server)(Chen, Chang, Kuo, & Heh, 2016; Chen, Kuo, Chang, & Heh, 2009; Chen, Kuo, Chang, & Heh, 2017). When a user signs into Moodle, he or she might want to see what in-game cards he or she has collected in the Trading Card Game. If the user has never authorized Moodle to access his or her data in the game, he or she needs to initiate the permission grant request through the Service Request module of ERIC API (see Steps 1 to 3 in Figure 2). ERIC API first generates a unique ID for the user (i.e., the client user uuid) and saves it into the ERIC API database (see Step 4). The Service Request module then sends the pre-registered information of Moodle (i.e., the client uuid) and the client user uuid to the Permission Granting module at the server side and asks the user to sign in at the service provider (i.e., the Trading Card Game) with their credentials there (see Steps 5 and 6).

Since the user is signing in at the service provider’s side, Moodle (as the client) is unable to know the user’s credentials of the Trading Card Game at all. After the user signs in, he or she needs to choose which permissions (e.g., retrieve the information of cards they collected or give the reward they got) he or she would like to authorize for Moodle. The information will be saved by the Authentication Process module (see Steps 7 and 8) and forwarded back to the Client Verification module at the client side (see Step 9). As soon as the authentication process is success and complete, the information will be saved into the ERIC API’s database at the client side (see Step 10).

2.3.   Integration of Moodle and Trading Card Game

In this section, we use a case to explain how a student grants Moodle to give him or her cards as rewards and retrieve the information of the cards that he or she has in the Trading Card Game via ERIC API. As Figure 3 shows, the student is aware of having a reward for the Math activity that he or she has done. To allow Moodle to give him or her the reward in the Trading Card Game, the student needs to click the “Trading Card Game” button in the “My Reward” Moodle block that uses ERIC API to bridging with the Trading Card Game.

Figure 3. "My Reward Moodle" block for students seeing the in-game cards they have been awarded.

After the student clicks the “Trading Card Game” button, the ERIC API redirects the student to the Permission Granting module at the Trading Card Game. As Figure 4 shows, the student will find that he or she is at the game’s website (i.e., the website’s address is tcg.is-ver-ood.org and now Trading Card Game can be accessed by the public at https://tcg.game-server.ca) instead of Moodle website (i.e., 192.168.56.101 shown in Figure 3 earlier) so he or she can feel comfortable to enter their credentials of the game and choose the permissions that he or she wants to grant for Moodle.

Figure 4. Permission granting page at the Trading Card Game server.

After the student’s credentials are verified, the student will be redirected back to Moodle. This time, the student can clearly see what in-game card as reward that he or she has got for the learning activities as Figure 5 shows.

Figure 5. The My Reward block now can show the details of the given reward.

 

3.   MATERIALS AND METHODS

3.1.   Hypotheses and the Study

User Experience (UX) starts playing important role in designing new technology in education (Nakamura, Marques, Rivero, de Oliveira, & Conte, 2019; Zaharias & Pappas, 2016). This research wants to use UX to know how students perceive the use of ERIC API in the learning management system to integrate the other system so they can review the educational rewards they are awarded. Based on the findings in this research, we might be able to figure out what types of students have no burdens in using such integrated system (with ERIC API) so we could use the cards in Trading Card Game to engage those students in learning easier and more efficient. The research findings could also help the research team to find out what factors in the design would make students have better acceptance of having two systems integrated with ERIC API so the team could revise the system design in order to engage more students in terms of using the integrated system with ERIC API for their learning.

Based on the reasons mentioned above, the research team wants to find the answer of the research question “what factors will affect students’ satisfaction toward the ERIC API?” To answer this research question, the research team creates the following research model and has five hypotheses and four moderators as Figure 6 shows.

Figure 6. Hypotheses of students’ satisfaction toward ERIC API.

The hypotheses are

·         H1: Students’ perceived effectiveness of the integration of Moodle and Trading Card Game (Moodle-TCG integration for short) with ERIC API has positive relation with their perceived usability toward Moodle-TCG integration.

·         H2: Students’ perceived efficiency of ERIC API has positive relation with their perceived usability toward Moodle-TCG integration.

·         H3: Students’ perceived effectiveness of Moodle-TCG integration has positive relation with their satisfaction toward the use of ERIC API to integrate systems.

·         H4: Students’ perceived efficiency of ERIC API has positive relation with their satisfaction toward the use of ERIC API to integrate systems.

·         H5: Students’ perceived usability toward Moodle-TCG integration has positive relation with their satisfaction toward the use of ERIC API to integrate systems.

The moderators are

·         Gender: is used to understand a student is male or female.

·         Experience in Technology: is used to understand their past experience in technology. The questions in the moderator are categorized into three groups. The first group questions ask users whether or not they have heard of the techniques (e.g., PHP, OAuth, etc.) that has been used or adopted by the ERIC API. There are eight techniques are asked in the questionnaire.

The second group of the questions asks students whether or not they have used Android app, Facebook, or using Facebook/Google/LinkedIn account to login in other websites since the workflows and experiences of using systems integrated with ERIC API are similar.

The last group of question is used to understand whether the users have experience in developing system with various techniques (e.g., PHP, Android app, etc.). There are seven techniques involved in the questionnaire.

·         Experience in Moodle: is used to understand their past experience with Moodle, including whether or not they have heard of Moodle and used Moodle before.

·         Experience in TCG: is used to understand their past experience with any trading card games. The question includes “I have heard any trading card game,” “I have played any trading card game,” and “I have seen others playing any trading card games.”

This research adopts System Usability Scale (SUS) designed by Brooke (1996). We also add several questions that are corresponding to the three factors, Effectiveness, Efficiency, and Satisfaction, from proposed by Lu (2011). Thirty-four five-point Likert-scale items (5 for “Strongly Agree” to 1 for “Strongly Disagree”) are included in the questionnaire for students who participated in this pilot. In Effectiveness factor, there are fifty items in total, including seven original SUS items and eight items adopted and altered from Lu’s research. The five items in Efficiency factors are also adopted and altered from Lu’s research. The last factor, Satisfaction factor, has fourteen items in which three of them are from SUS and the rest are from Lu’s research. 

 

3.2.   The Collected Data

The research team recruited students from a course given by the Department of Information Management in a university in northern Taiwan in 2018 Spring semester. Twenty-six students were recruited, including 7 males and 19 females. Although the size of this pilot study is small, the number of sample size is sufficient for the Human Computer Interaction studies (Hwang & Salvendy, 2010; Nielsen, 2012). In the study, the research team first demonstrated how students grant permission for Moodle with the ERIC API enabled Moodle block to access the cards they collected in the Trading Card Game. Following with the demonstration, the students are given time to try on the block themselves. In the end, the research team asked the students to fill out a questionnaire regarding their demography, experience in technology/Moodle/TCG, perceived effectiveness and efficiency of ERIC API, and satisfaction toward the integration system.

With the collected data, the research team first investigated the students’ technology background and found that only one user has never heard the eight technologies (PHP, JSP, ASP, JavaScript, HTML, SQL, Mobile app, and OAuth). In average, the students know at least five technologies on the list (mean value Mean = 5.1 and standard deviation SD = 1.73). There is also one student has never used Android app, Facebook, nor using Facebook/Google/LinkedIn account to logon other websites. Three of the students have no experience in system development. Overall, the students have used at least two out of seven technologies mentioned in the questionnaire (Mean = 2.4 ad SD = 1.65).

Table 1 summarizes students’ experience with Moodle. Half of the students have heard what Moodle is, but only seven (26.92%) have used Moodle before. There is no gender difference in the experience with Moodle. The result of chi-square test of gender in “I have heard what Moodle is” question is χ² (1, N = 26) = 1.759, p < .189 and “I have used Moodle before” question is χ² (1, N = 26) = 0.778, p < .365.

Table 1. Descriptive statistics of students’ experience with Moodle.

 

Regarding experience with trading card games, most of the students (80.77%) have heard what trading card game is as well as have seen others playing trading card games (84.62%) as the summary listed in Table 2. There is also no gender difference in both questions. The chi-square test for gender in “I have heard what trading card game is” is χ² (1, N = 26) = 0.151, p < .589 and “I have seen others playing trading card games” is χ² (1, N = 26) = 0.009, p < .713. However, there is a gender difference found for the question “I have played any trading card games” – χ² (1, N = 26) = 7.394, p < .001. There are 75.71% of male students have played trading card games before, but only 26.32% of female students have that experience.

Table 2. Descriptive statistics of students’ experience with trading card games.

 

3.3.   Validity and Reliability

The research team first calculated the usability score based on the Brooke’s equations (https://uiuxtrend.com/measuring-system-usability-scale-sus/). The usability score’s value should be falling between 0 to 100. After that, the research team reversed the five negative worded SUS items before doing further validity and reliability tests. The research team used SPSS 20.0 to verify the questionnaire and the collected data’s validity and reliability. Fourteen items had to be removed because of their low contribution toward the factors. The overall Cronbach’s alpha value of the remaining twenty items is 0.928, which sits in the “excellent” range and shows that questionnaire is reliable (Georage & Mallery, 2010). Table 3 lists the analysis result of Effectiveness factor: two sub-factors are discovered, which are Ease of Learn (EoL) and Ease of Use (EoU). Table 4, on the other hand, lists the analysis results for the Efficiency factor.

Table 3. Validity analysis result for the Effectiveness factor

 

Table 4. Validity analysis result for the Efficiency factor

 

There are also two sub-factors found in the analysis results for the Satisfaction factor as Table 5 listed.

Table 5. Validity analysis result for the Satisfaction factor

 

 

4.   ANALYSIS

4.1.   Hypotheses Verification

In the beginning, the research team calculates the mean values for each factor and sub-factor as well as the system usability score based on the equations. Table 6 lists the mean values and standard deviations for each factor and sub-factors as well as the system usability score that student gave for the Moodle-TCG integration. The average perceived Effectiveness that students have toward the Moodle-TCG integration is below Neutral but the average perceived Efficiency and Satisfaction toward ERIC API and the Moodle-TCG integration are more positive. The average of participants’ response in SUS score is 54.23. Based on Sauro’s research in 2011, the average score of SUS is 68; the result indicates that the SUS score in Moodle-TCG integration is below the average.

Table 6. Mean and standard deviation of students’ perceptions toward the integration of Moodle-TCG and ERIC API

 

The research team verifies the hypotheses H1 to H4 described in Figure 6 with Pearson’s correlation coefficient and the analysis results are listed in Table 7. Both H1 are H2 are proved – the perceived Effectiveness toward the Moodle-TCG integration and the perceived Efficiency for ERIC API has positive correlation with the students’ perceived usability of Moodle-TCG integration. The results indicate that students who consider the Moodle-TCG integration is effective or feel ERIC API can make their works more efficiently score the usability of Moodle-TCG integration higher. However, the perceived Ease of Use toward the Moodle-TCG integration has negative correlation with the perceived usability.

Table 7. The Pearson correlation analysis results for finding correlations among factors and usability score.

^*: p < 0.05, ^**: p < 0.01.

 

On the other hand, both perceived Effectiveness toward the Moodle-TCG integration and perceived Efficiency for ERIC API have found positive correlation with the Satisfaction of ERIC API and the Moodle-TCG integration. In Effectiveness factor, the perceived Ease of Use does not show significant correlation with the perceived Satisfaction. The result shows that when students feel the Moodle-TCG integration is easy to learn, there is good opportunity for them also being more satisfied with it as well as ERIC API.

The hypothesis H5 is also verified. The Pearson Correlation Coefficient between usability score and Satisfaction factor is r = .530, p = 0.005. Moreover, the Pearson Correlation Coefficient between the usability score and the two sub-factors of Satisfaction (Moodle-TCG Integration and ERIC API) are r = .427, p = .030 and r = .556, p = .003 respectively. This result is in line with the assumption of when a user is more satisfying with a system he or she may score it higher.

4.2.   Moderators

The research team then analyzes which moderator will make perceived Effectiveness, Efficiency, Satisfaction, and Usability different. The first moderator we evaluate is gender. Table 8 lists the independent t-test results for different genders’ perceived Effectiveness, Efficiency, Satisfaction, and usability score. The results show that gender has no influence on a student’s perceptions toward the Moodle-TCG integration and ERIC API.

 

Table 8. Independent t-test result for different gender’s Effectiveness, Efficiency, Satisfaction, and Usability.

 

The research team uses Pearson Correlation Coefficient to check the correlation between students’ technology background and their perceptions as Table 9 shows. The results indicate that students’ technology background also not make their perceived efficiency toward ERIC API and perceived satisfaction different. On the other hand, students who have known more technologies or capable of developing systems in more programming languages have positive relation with perceived Ease of Learn and usability score. However, participants who have used more different technologies have negative relation with perceived Ease of Use.

Table 9. The Pearson correlation analysis for students’ technology background and perceptions.

 

Based on the t-test results listed in Table 10, there is no significant difference between students’ Moodle experience and their perceptions. The results indicate even when a student has never used Moodle before, he or she will still like to use the Moodle-TCG integration and rate ERIC API or the Moodle-TCG integration positively.

Table 10. Independent t-test result for Effectiveness, Efficiency, Satisfaction, and Usability in Moodle usage.

 

As Table 11 shows, the t-test is also used to find out whether there is a significant difference between students’ experience in any trading card games and their perceptions. The results show that there is significant difference in the given usability scores between students who have and have not heard any trading card games as well as who have seen and have not seen others playing any trading card games. Students who have heard any trading card games or have seen others playing any trading card games give higher Usability scores.

Table 11. Independent t-test result for Effectiveness, Efficiency, Satisfaction, and Usability in any trading card game experience.

 

The analysis also finds out that there is a significant difference for the perceived Efficiency toward ERIC API between students who have and have not played any trading card games. The result indicates that students who have never played any trading card games have more positive perceptions on the Efficiency of ERIC API. Moreover, there is also a significant difference for perceived Ease of Use between students who have seen and have never seen others playing any trading card games. The result shows that students who have never seen others playing any trading card games have more positive perceptions toward Ease of Use Moodle-TCG integration.

 

5.   RESULTS AND DISCUSSION

This section lists the information reveals from the analysis results described in the previous section and categorizes the findings into three kinds: common findings, important findings and unexpected findings.

5.1.   Common Findings

The five hypotheses proposed in Figure 6 are verified through the Pearson correlation analysis. The results show that when students perceive positive on the effectiveness of the integration of Moodle and Trading Card Game and the efficiency for them checking out the rewards they have received from what they have done for the learning activities, they will have higher intention of using more ERIC API enabled integrated systems. Moreover, they also have higher intention of having cards in Trading Card Game as educational rewards in other courses later.

On the other hands, when students find out the use of ERIC API can integrate two systems easily, they might want to use it to integrate other systems in the future in their jobs. For example, the enrollment system and learning management system that a university uses are usually two independent systems. Students might only use the enrollment system once or twice a semester but signing into the learning management system almost every day. If students can review their enrollment information inside the learning management system without further signing the enrollment system separately but only authorize the permission for the learning management system once (and can revoke the permission granted at any time they want), the convenience of seeing and checking everything at a single platform can not only help them understand their status in the journey but make them capable of planning and thinking their next steps – what courses will be offered and which should they register for next semester according to their progress in the current one? 

5.2.   Important Findings

Although the five proposed hypotheses are verified, the system usability score of the Moodle-TCG integration in this study is only 54.23, below the average score 68. However, based on Bangor and colleagues’ finding (Bangor, Kortum, & Miller, 2009), the SUS score can also classified as best imaginable, excellent, good, ok, poor, and worst imaginable as Figure 7 shows. The usability score 54.23 indicates that students might still think the integration of Moodle and Trading Card Game is acceptable for using. To find out how to improve the usability of the Moodle-TCG integration, the research team investigates the potential factors that might influence the usability score. From the common findings described earlier, the effectiveness and efficiency factors have positive correlation with SUS score; therefore, we would like to take a deeper look to the data.

Figure 7. Adjective rating in SUS Score (Bangor, Kortum, & Miller, 2009)

First of all, we find out that the ease of learn in the Moodle-TCG integration has positive relation to students’ satisfaction on Moodle-TCG integration and ERIC API usage as well as their perceived usability toward the Moodle-TCG integration. On the other hand, students’ technology background is the only moderator that will affect their perceived ease of learn. The analysis results show that students who are capable of developing systems with more programming languages believe the process of authorizing Moodle to access their data in Trading Card Game is easy to learn. However, neither all of the students in this study nor all of the users in the world are familiar with programing languages and are system developers. Therefore, designing an easy to learn user interface for the Moodle-TCG integration becomes an important issue.

Another interesting finding is no gender difference has been found for students’ given SUS score. Some studies show that male participants have higher acceptance in learning technologies (Ong & Lai, 2006). However, more and more studies show that there is no gender difference in terms of using new technologies (Volman, Van Eck, Heemskerk, & Kuiper, 2005; Wang, Wu, & Wang, 2009); sometimes, even females have higher acceptance in adopting technologies for learning (Arbaugh, 2000; Fan, Kuo, Chang, & Heh, 2015; Viberg & Gronlund, 2013). This study’s result shows that as long as the system is useful, both male and female students would like to access the information of in-game card reward that they received in Trading Card Game from Moodle. The result leads to another question: can we use in-game cards of the Trading Card Game to engage students learning?

Based on the analysis results listed in Table 11, students who have heard trading card games or have seen their friends playing trading card games give higher scores for the usability of the Moodle-TCG integration. The results might tell us that students who know what trading card game is may have more interests in having in-game cards of the Trading Card Game as educational rewards. When the Trading Card Game is useful as the educational rewards, the integration between Moodle and Trading Card Game can get students motivated to check the information of the rewards they received from Moodle. To understand whether this hypothesis is correct or not, the research team may need to conduct a more complete and longitude experiment.  

5.3.   Unexpected Findings

Although students who have heard trading card games or have seen others playing trading card games have more interests in using the Moodle-TCG integration, students who have played trading card games before does not have more interests in the use of the Moodle-TCG integration as the results listed in Table 11 shows. The research team investigates it further and found that “students who have played trading card games have perceived less positive on the Efficiency of ERIC API. One possible reason is they have experience in playing commercial trading card games and are used to play games without the need of authorizing permission for another system. The permission grant and access control process that they experienced when they first used the system might confuse them and turn out to be less positive on the perceived Efficiency toward ERIC API.

However, the main purpose of the ERIC API is integrating two systems with preventing private information leakage; the permission grant steps at very beginning before two systems can exchange data safely are required. Based on students’ response, none of them have heard OAuth technology which is similar to the technology used in ERIC API. Therefore, we are unable to know whether students’ knowledge in cybersecurity make things. Taking students’ computer security awareness and understanding into consideration as moderator would be necessary for the future work.

Another unexpected finding is that students who believe the Moodle-TCG integration is easy to use actually has more negative perceptions in its usability. One of the possible reason is that all of three questions in the Ease of Use sub-factor are negative worded. Negative questions usually give reader more cognitive loading (Kamoen, Holleman, Mak, Sanders, & Van Den Berh, 2017). The negative worded questions might make students unable to answer the question properly. It might also be the cause of other unexpected findings – “students who have never seen others playing any trading card games have more positive perceptions toward the ease of use of the Moodle-TCG integration” as well as “students who have used more different technologies have negative relation with perceived Ease of Use.” Since negative worded questions may not be able to prevent response bias (Van Sonderen, Sanderman, & Coyne, 2013), the research team should consider using positive worded question to investigate participants’ perceived ease of use.

 

6.   CONCLUSION

This research evaluates the usability of the Moodle-TCG integration with ERIC API. The results show that the usability score is acceptable for users have less knowledge of computer security. The ease of learn plays an important role in the Moodle-TCG integration; students who believe the process of using the Moodle-TCG integration is easy to learn have more positive satisfaction toward the integration. Moreover, students who have heard trading card games or have seen others playing trading cards have more interests in having in-game cards of the Trading Card Game as educational rewards.

There are some limitations of this study. First of all, the permission grant process in the Moodle-TCG integration is required for secure delegated access data without sharing credentials. It might make students who have played trading card games before perceive less efficiency. Moreover, the students participated in this study do not have knowledge in cybersecurity and we may not know whether or not students’ cybersecurity awareness affects their perceived usability and satisfaction. Another limitation of the study is the time spent. The pilot study only asked students to try on the Moodle-TCG integration and they don’t opportunity to see how useful ERIC API is for the integration task is in the real world with real cases.

To solve these issues, the research team would like to further design an easy-to-learn user interface for students granting permission for Moodle to access their data in the service provider (e.g., the Trading Card Game). We would like also to have a one to two months’ experience for students so they can receive and use the cards in Trading Card Game from time to time when they complete learning activities in Moodle. Last but not the least, we want to further investigate the influence of students’ computer security awareness like the study of Rounds and colleagues did in 2008 (Rounds, Pendegraft, Pendegraft, & Stone, 2008) and Tirumala and Sarrafzadeh did in 2016.

 

REFERENCES

Almeida, L. R., da Costa, J. P. C., de Sousa, R. T., de Freitas, E. P., Canedo, E. D., Prettz, J., Zacarias, E., & Galdo, G. D. Motivating attendee’s participation in distance learning via an automatic messaging plug for the moodle platform. 2016. In 2016 IEEE Frontiers in Education Conference (FIE), p.1-5, 2016.

Arbaugh, J. B. An exploratory study of the effects of gender on student learning and class participation in an Internet-based MBA course, Management Learning, v.31, n.4, p.503-519, 2000.

Bangor, A., Kortum, P., & Miller, J. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies, v.4, n.3, p.114-123, 2009.

Brooke, J. SUS-A quick and dirty usability scale. Usability evaluation in industry, v.189, n.194, p.4-7, 1996.

Charleer, S., Santos, J. L., Klerkx, J., & Duval, E. Improving teacher awareness through activity, badge and content visualizations. Lecture Notes in Computer Science, v.8699, p.143-152, 2014.

Chen, C.-L., Chang, M., & Chang, H.-Y. Educational Resource Information Communication API (ERIC API): The Case of Moodle and Online Tests System Integration. 2016. In Proceedings of the 3rd International Conference on Smart Learning Environment, p.225-229, 2016.

Chen, C.-L., Zhao, Y, Luo, A., Chang, M., Qian, D., Kuo, R., & Chang, H.-Y. Educational Reward Moodle Plug-In. In Proceedings of 21st Global Chinese Conference on Computers in Education (GCCCE 2017), p.211-218, 2017.

Chen, P., Chang, M., Kuo, R., & Heh, J.-S. Trading Card Game. 2016. In the Proceedings of the 24th International Conference on Computers in Education (ICCE), 2016. p.6-11.

Chen, P., Kuo, R., Chang, M., & Heh, J.-S. Designing a Trading Card Game as Educational Reward System to Improve Students’ Learning Motivations Transaction on Edutainment, v.III, p.116-128. Berlin: Springer-Verlag, 2009.

Chen, P., Kuo, R., Chang, M., & Heh, J.-S. The Effectiveness of Using In-Game Cards as Reward. Research and Practice in Technology Enhanced Learning, v.12, Article #15, 2017.

Cohen, J. Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, 1988.

Fan, M. X., Kuo, R., Chang, M., & Heh, J. S. Story-Based Virtual Experiment Environment. Smart Learning Environments. Springer, Berlin, Heidelberg, v.10, p.175-198, 2015.

Ferry, R., Raw, J. O., & Curran, K. Security evaluation of the OAuth 2.0 framework. Information & Computer Security, v.23, n.1, p.73-101, 2015.

George, D., & Mallery, P. SPSS for windows step by step: A simple guide and reference 18.0 update (11th ed.). Boston, MA: Allyn & Bacon, 2010.

Graf, S. & Kinshuk. Dynamic Student Modelling of Learning Styles for Advanced Adaptivity in Learning Management Systems. International Journal of Information Systems and Social Change, v.4, n.1, p.85-100, 2013.

Hwang, W. and Salvendy, G. Number of people required for usability evaluation: The 10±2 rule. Communications of the ACM, v.53, n.5, p.130–133, 2010.

Jurubescu, T. Learning Content Management Systems. Informatica Economica, v.4, n.48, p.91-94, 2008.

Kamoen, N., Holleman, B., Mak, P., Sanders, T., & Van Den Bergh, H. Why are negative questions difficult to answer? On the processing of linguistic contrasts in surveys. Public Opinion Quarterly, v.81, n.3, p.613-635, 2017.

Kardara, M., Kalogirou, V., Papaoikonomou, A., Varvarigou, T., & Tserpes, K. SocIoS API: A data aggregator for accessing user generated content from online social networks. 2014. In the Proceeding of the 15th International Conference on Web Information System Engineering (WISE 2014), p.93-104, 2014. Retrieved from http://arxiv.org/ftp/arxiv/papers/1505/1505.02977.pdf

Kasim, N. N. M., & Khalid F. Choosing the Right Learning Management System (LMS) for the Higher Education Institution Context: A Systematic Review. International Journal of Emerging Technologies in Learning, v.11, n.6, p.55-61, 2016.

Lu, C. Story Decorated Context-Awareness Role Playing Learning Activity Generation. 2011. M.S. Thesis in School of Computing and Information Systems, Athabasca Univ., Athabasca, Canada, 2011.

Nakamura, W. T., Marques, L. C., Rivero, L., de Oliveira, E. H., & Conte, T. Are scale-based techniques enough for learners to convey their UX when using a Learning Management System? Brazilian Journal of Computers in Education, v.27, n.1, p.104-131, 2019.

Nielsen, J. How many test users in a usability study? 2012. Retrieved 21 May 2019, from https://www.nngroup.com/articles/how-many-test-users/

Ong, C. S., & Lai, J. Y. Gender differences in perceptions and relationships among dominants of e-learning acceptance. Computers in human behavior, v.22, n.5, p.816-829, 2006.

Rounds, M., Pendegraft, R., Pendegraft, N., & Stone, R. Student survey on computer security awareness and responsiveness. May, 2008. In CONF-IRM 2008 Proceedings. p. 48, 2008.

Sauro, J. A practical guide to the System Usability Scale: Background, benchmarks, & best practices. Denver, CO: Measuring Usability LLC, 2011.

Tirumala, S., & Sarrafzadeh, A. A survey on Internet usage and cybersecurity awareness in students. Sarrafzadeh, A. (Ed.), December, 2016. In The 14th Privacy, Security and Trust Conference, p.223-228, 2016.

Torre, L., Sanchez, J. Andrade, T. F., & Restivo, M. T. Easy creation and deployment of Javascript remote labs with EjsS and Moodle, 2016. In The 13th International Conference on Remote Engineering and Virtual Instrumentation (REV), p.260-261, 2016.

Van Sonderen, E., Sanderman, R., & Coyne, J. C. Ineffectiveness of reverse wording of questionnaire items: Let’s learn from cows in the rain. PloS one, v.8, n.7, e68967, 2013.

Viberg, O., and Gronlund, A. Cross-cultural analysis of users' attitudes toward the use of mobile devices in second and foreign language learning in higher education: A case from Sweden and China, Computers & Education, v.69, n.1, p.169-180, 2013.

Volman, M., Van Eck, E., Heemskerk, I., & Kuiper, E. New technologies, new differences. Gender and ethnic differences in pupils' use of ICT in primary and secondary education. Computers & Education, v.45, n.1, p.35-55, 2005.

Vozniuk, A., Rodriguez-Triana, M. J., Holzer, A., Govaerts, S., Sandoz, D., Gillet, D. Contextual Learning Analytics Apps to Create Awareness in Blended Inquiry Learning. 2015. Paper presented at The 14th International Conference on Information Technology Based Higher Education and Training (ITHET), p.1-5, 2015.

Wang, Y. S., Wu, M. C., & Wang, H. Y. Investigating the determinants and age and gender differences in the acceptance of mobile learning. British journal of educational technology, n.40, v.1, p.92-118, 2009.

Zaharias, P., & Pappas, C. Quality management of learning management systems: A user experience perspective. Current Issues in Emerging eLearning, n.3, v.1, Article 5, 2016.

 

 

 

 

 

 

  This work is licensed under a Creative Commons Attribution 4.0 International License.