International Journal of Scientific & Engineering Research Volume 4, Issue 2, February-2013 1
ISSN 2229-5518
INTELLIGENT AGENT BASED GRID
RESOURCE MANAGEMENT SYSTEM
P. Muthuchelvi, G.S.Anandha Mala
St. Joseph’s College of Engineering, Chennai, India
Tel – 988490 3640, 9884903040
Email – muthuchelvi69@gmail.com
Abstract - In grid environment, users look for a suitable resource capable of executing a given task. Resource brokers / schedulers are intermediaries, who provide service to the grid users. In the existing resource brokers / schedulers, when the exact match for the requested resource is not available, resource discovery have to be terminated unsuccessfully or the process should wait in the queue till the availability of the requested resources. This results in decrease in success rate of resource discovery and increases the waiting time for the processes. However, at the same time, there is a possibility of the existence of lot of almost similar, suitable and acceptable resources with negligible difference in the requirements, with which the user’s job can be executed, remain underutilized in the grid. In our earlier work, Resource brokers / schedulers were knowledge assisted by cognitive agent in order to provide alternative resource for expediting resource discovery. Cognitive agent is an intelligent agent and provides knowledge assistance to brokers. This paper focuses on the enhancement of the functionalities of cognitive agent. The proposed system has been found to produce a consistently higher success rate of resource discovery and improved resource utilization besides reducing the cost-loss for the requester that leads to better performance of the grid.
Index Terms: Grid, Agents and Grid resource management
Grid is a system that Coordinates resources that are not subject to centralized control, Uses standard, open, general- purpose protocols and interfaces, and Delivers non-trivial Qualities of Services. The main objective of grid computing is resource sharing through an efficient resource management system. In spite of current developments in grid computing, many issues and challenges in grid environment such as resource discovery, sharing, security etc., are yet to be analyzed and solved. The most important challenges in grid resource management include intelligent resource discovery, resource selection, resource allocation, scheduling, quality of services etc. The proposed work is to provide knowledge assistance by the intelligent agent, which increases the resource utilization, success rate of resource discovery and profit to the resource providers. The proposed system in this paper helps the resource brokers to handle the dynamic nature of the grid.
Buyya (2000) proposed economy driven resource management architecture for global computational grids. This model consists of a generic framework, called GRACE (Grid Architecture for Computational Economy) for dynamic trading resources, in conjunction with existing grid components such as local and global grid schedulers. This approach sorts to enable the supply and demand of resources to regulate and control access of computational resources.
Chee Shin Yeo (2005) proposed a system which
incorporates Service Level Agreements (SLA) while providing service to the grid users. SLA is prepared between resource providers and requesters. They considered penalty for any deviation in SLA into the admission control and scheduling decisions in a cluster.
Dominiak et al (2006) proposed a new approach based on agent teams to facilitate resource discovery with the yellow page services. The master agent is used to select a team to execute a job. The method used in the approach is not feasible in a dynamic grid situation. The proposed infrastructure does not, however, reckon the full potential of grids which are of highly dynamic nature besides using only a single service provider. Birje et al (2006) presented an agent-based model for resource discovery, brokering and allocation of cost effective resources to computational jobs. A wireless grid environment of virtual organizations is considered. Agents are used to perform resource brokering and allocation tasks. The grid environment uses a cluster based agent framework and it supports faster discovery of resources in wirelessgrids.
Somasundaram et al (2007) described a scheduling method
for efficient utilization of grid resource using next highest
response scheduling. The method provides more response
with time, memory and CPU requirements. It is adaptive
for local and remote jobs without any loss of performance and also highly adaptive for grid environment. Junyan Wang et al (2008) presented a new resource discovery mechanism with negotiate solution based on agent in grid environments. They considered pricing fluctuation manager in order to improve the efficiency of the resource management. Feedback capability is used to adapt the highly dynamic grid environment. Quan liu et al (2009) described grouping a based fine grained job scheduling in grid computing. In their work, the fine grained jobs are grouped into forming coarse grained are allocated to the available resources according to their processing capabilities. The grouping algorithm integrates greedy algorithm and first come first serve algorithm to improve the processing of fine grained jobs.
IJSER © 2013 http://www.ijser.org
International Journal of Scientific & Engineering Research Volume 4, Issue 2, February-2013 2
ISSN 2229-5518
Sim (2010) presented a survey and new direction of grid resource negotiation in grid resource management. He discusses most of the works in the grid negotiation and mentioned that many of the bargaining models do not involve third party negotiation and trade-off-analysis is required to solve the difference of opinions of grid users. He also discussed new directions in grid resource negotiation.
Raksha Sharma et al (2010) presented a survey of job
scheduling and resource management in grid computing. A
comparison on various parameters like distributed,
hierarchical, centralized, response time, load balancing and resource utilization is reported. They consider memory requirements of the job while submitting the jobs to the selected resources.
Though there are many research works focusing in this area, there are still many issues meriting consideration. Further, there is no knowledge assistance to manage the vast infrastructure of the grid. Considering the inadequacies of the efforts made earlier and referred to
above, the motivation behind the present work is to develop an intelligent agent-based resource management
system for devising a speedy and very efficient method of resource discovery in a grid environment. The proposed system envisages association of cognitive agent for knowledge assistance to strengthen the work. This paper focuses on the convergence of the agent and the grid technologies.
The generalized architecture of the proposed Grid Resource Management System is shown in Figure 3.1. The components of the proposed system comprise grid users, Resource Requester Agents (RRA), Resource Provider Agent (RPA), Negotiation and Alternative Solution Provider (NASP) agent, NFR preferences elicitor and Cognitive agent. These agents are proposed for the efficient management of resources to the advantage of resource providers and requesters in ensuring speedy execution of processes. Functions of these agents are discussed in [Muthuchelvi et al 2009]. A function of cognitive agent is further extended in this paper.
Figure 3.1 Architecture of Grid Resource Management System
The developments of the grid in recent times have significantly increased its performance. But still, the position remains inadequate. One possible reason for this
could be that the presently available resource brokers / schedulers do not have any intelligent support with them. But, grid has very large infrastructure, in order to manage such a large infrastructure efficiently and judiciously, there must be knowledge assistance. The cognitive agent acts as a
IJSER © 2013 http://www.ijser.org
International Journal of Scientific & Engineering Research Volume 4, Issue 2, February-2013 3
ISSN 2229-5518
back-up assistant for providing information when needed. It acquires its knowledge from the processes happening in the system and updates itself for fulfilling its role. Cognitive agent is thus a customized agent in the proposed system responsible for the following services:
In the dynamic grid environment, different types of resource providers may offer different types of resources to the grid. These resources are a mix of high / low demand resources. High and low demands are based on many factors such as the nature of resource, cost of resource and availability of resource. The nature of resource may be ordinary or special purpose resource. Whereas, the availability of resources means some resources may be available only in the specific period of time. In a grid market, the user faces different scenarios in different occasions with different types of resources. Demands depend on the performance, availability, and cost of resources. In addition, demands also depend on the reliability and trust-worthy nature of the provider. Thus, in order to manage the dynamic nature, resource brokers are assisted by the cognitive agent. Demand factor of the resource is estimated for the purpose of identifying highly
/ less demanded resource,
Demand factor = number of requests for a specific resource/ total number of requests (1)
Resource request (res-request) is placed according to the user requirement, interest, preferences and resource constraints, where (res-request) is the subset of (res- available). Sometimes most of the resources in the list could meet the requirements stipulated by the requester. So there must be some mechanism to choose the best resource for the job submitted. Although random selection is a choice, it is not ideal. If the broker has knowledge assistance for identifying a suitable resource, an ideal solution may be provided. In the proposed work, the broker is knowledge assisted to select a suitable resource with the help of cognitive agent.
When the list is displayed, the user may / may not have a precise knowledge of the most suited resource for executing the specified application. If he has adequate knowledge, the resource is selected and bypasses the scheduler and assigns the resource to the job. If the user does not have any knowledge about the resource, there is no known approach to help the user in finding a suitable
Success rate is used to find the performance of resource providers, requesters and resource. The requester needs to get the resources from the best providers. At the same time, providers need to give their resource to the best requester. In the proposed system, success rate of the provider (Sr-p) is initially assigned by the brokers based on the resource information that is provided. Thereafter, genuine feedback from the requester is considered to find out success rate of that particular provider. This feedback includes
completion of the assigned job (c)
completion of job within assigned period
(t) and
Completion of job within a assigned
budget (b).
The success rate of the provider from the requesters’ feedback (Sr-pr) for particular transaction is found out as follows;
Sr-pr = x1*c + x2*t +x3*b (2) where x1, x2 and x3 are equal weights assigned to c, t and b respectively. Success rate of the provider is based on the broker’s (NASP) feedback also. A broker has information about the behavior of the providers, for example whether
committed resource (r) is provided
committed network bandwidth (n) is provided
provider maintains good relationship (s)
with brokers.
The success rate of the provider from the feedback (Sr-pnasp) for particular transaction is found as follows:
Sr-pnasp = y1*r + y2*n + y3* s (3) where y1, y2 and y3 are equal weights assigned to r,n and s respectively. so the success rate of the provider is the average success rate of requester and brokers given below:
Sr-p = (Sr-pr + Sr-pnasp) / 2 (4) The average success rate of provider after n transactions is computed using the following equation.
n
Sr p / n
resource for the special purpose application. Also, it is
Av-Sr-p = i 1
(5)
impossible that all the users are aware of special purpose resource. In such scenario, the user gets assistance from the cognitive agent to execute the job.
The success rate of the resource is estimated with the help of the utilization report generated by the requester to broker. This report reflects the satisfaction level of the user. The success rate of the requester is based on the behaviors
IJSER © 2013 http://www.ijser.org
International Journal of Scientific & Engineering Research Volume 4, Issue 2, February-2013 4
ISSN 2229-5518
like usage of the resource (u) (should not misuse the resource) , prompt payment for the usage of resource (p) flexibility of accepting the price for alternative resource (f) The success rate of the requester for a particular transaction is computed using the following equation
Sr-req = w1*u + w2*p + w3*f (6) where w1, w2 and w3 are equal weights assigned to u, p and f respectively. The values of u. p and f are obtained from the providers or broker agent as it is the mediator between the providers and requesters. The average success rate of the requester after ‘n’ transactions is computed as follows
n
Sr req / n
Providers have their own resource policies like cost, time, availability etc. Generally in order to increase the profit, before setting up the price of that particular resource, providers wish to know about the competitor’s price for that particular resource. When the providers try to register their resource with brokers, it is human curiosity to know about other provider’s price. In such a case, they can get assistance from the cognitive agent to display the cost of other providers for the same type of resource. Some providers may fix the price for their resources based on the others’ prices. To attract more users and to increase the profit, initially providers may fix the lower price than others. After some time, they may increase the cost.Some requesters wish to look at the cost of all the resources before making a request for the resources to have an idea
about the cost of resources. If the user comes to know about
Av- Sr-req =
i 1
(7)
the cost of the resource before placing the request, there is a
In the system, performance evaluation is based on the
experience of RRA after utilization of the resource. This
approach helps to find the better suited resource when
more number of matches for the request is available. The same procedure is followed, also for finding out the best resource provider and resource requester.
When the resource provider tries to register the resource to the broker, information regarding infrastructure and policies are also given. Based on this information, priority is assigned to the particular resource provider by the brokers. Similarly priority is assigned for all the providers initially. This priority assignment is purely based on the information given by the provider itself. Thereafter priority is updated after the completion of each transaction assigned to the particular resource provider. The cognitive agent provides the details of available providers in the order of priority
possibility of placing the order according to the budget or to increase the budget according to the cost of resource. When the high demand resource required in intense pressure, user may offer a price which is more than the cost of resource. So that resource is assigned to a requester who is ready to pay more than the cost of resource.
Grid architecture shown in Figure 4.1 is used to identify the fundamental system components, purpose and functions of these components in grid environment. A grid environment is created using grid middleware - Globus Toolkit Auto Install utility (GTAI 1.0.1). All the prerequisite software such as J2SE, Ant, Tomcat and Junit are bundled in GTAI package. Agents are created using JADE framework. The agent platform has been split on several hosts provided. Agents communicate by using FIPA-request interaction protocol
Command
Line Interface
Portals Agents Grid Resource Registration
Application layer
Repository
Intelligent agent based grid resource management
Collective layer
Network
Monitoring Tool
Meta
Schedulers
International Journal of Scientific & Engineering Research Volume 4, Issue 2, February-2013 5
ISSN 2229-5518
Figure 4.1 Deployment of agents in the grid architecture
The architecture shows how these components interact with one another to achieve the goal. It has many protocols, services, application programming interfaces and software development kits to enable effective resource sharing. Generally grid architecture has five layers namely fabric layer, connectivity layer, resource layer, collective layer and application layer (Foster et al 2001).
The fabric layer provides the resources available in grid. Such resources may be of various types like hardware resources, software resources and the resources to satisfy QoS requirements etc.Resource and connectivity layers play a vital role in a grid like monitoring and discovery of grid resources, execution of job in grid resources and file transfer between grid resources. In order to perform these tasks, Grid middleware incorporates the necessary components like grid information services (MDS), grid execution services (GRAM) and grid file transfer services (Gridftp, RFT). These services are utilized by the Meta- schedulers or brokers through Middleware Access Drivers (MAD).
The collective layer is responsible for resource brokering and scheduling. Resource brokers perform matchmaking and schedulers allocate the job to the best resource. The existing resource brokers / schedulers (gridway) has the
provision for adding external plugins to enhance its role. The system is added as the plugin to improve the performance.
When the requested resource is not available, instead of adding the job to the waiting queue, the resource broker provides an alternative resource. This is accomplished by disabling the built-in engine policy in the configuration file and enabling the agent-based resource management system proposed in this paper. The user specifies the exact resource name for specific application, the functions of schedulers is bypassed by changing the configuration file of brokers. The application layer enables the use of resources in a grid environment through several portals.
Figure 4.2 shows the percentage of resource utilization Vs number of processes submitted. This graph shows that the percentage of resource utilization is always higher when the system is knowledge assisted than without knowledge assisted. It is observed that the number of processes completed by resource brokers with the proposed system is constantly higher than the processes completed earlier. The increase in resource utilization is also ascertained when the proposed system is knowledge assisted by the cognitive agent.
IJSER © 2013 http://www.ijser.org
International Journal of Scientific & Engineering Research Volume 4, Issue 2, February-2013 6
ISSN 2229-5518
Figure 4.2 Number of processes Vs percentage of utilization of resources
Thus, with the proposed integrated system, resource brokers provide efficient, knowledge assisted, cost-effective and speedy management of resources. The system is tested with hundred resource requesters and providers. Cost-loss of requesters against number of transactions in order to select best provider is shown in Figure 4.3. In the system, cost-loss defined as cost spent for the unsatisfactory services. A level of satisfaction is assumed from the range 0 to 1. Cost-loss to requester graph is plotted against the number of transactions for the selection of resource providers with and without knowledge assistance.
As expected, cost-loss for the requesters is reduced when the requesters’ feedback is considered. Cost-loss is further reduced when the requesters as well as NASP feedback are considered. The simulated results show that selection of trustworthy resource provider leads to better performance with satisfaction of the requesters and less cost-loss for the requesters’ community.
IJSER © 2013 http://www.ijser.org
International Journal of Scientific & Engineering Research Volume 4, Issue 2, February-2013 7
ISSN 2229-5518
6000
5000
4000
3000
2000
1000
0
1 2 3 4 5 6 7 8 9 10
Without know ledge assistance
w ith know ledge assistance w ith requesters feedback only
w ith know ledge assistance w ith requesters and NASP
feedback
Figure 4.3 Cost-loss with requesters and NASP feedback
On numerous occasions, grid users face non-availability of high-end resources for completing the task on hand, even though many suitable resources are available as underutilized in the grid. It is in this context, proposed intelligent agent based system is shown to be robust in
.
REFERENCES
terms of providing alternative resource when resource discovery fails. Thus the system is more appropriate, convenient and plays a crucial role in bridging the wide gap between grid and agent technologies
[1]. Muthuchelvi P , Anandha Mala G.S. and Ramachandran V , “Agent Based Grid Resource Discovery with Negotiated Alternate Solution and Non-Functional Requirement Preferences “ Journal of Computer Science, “ 5 (3): pp: 191-198, 2009 ISSN 1549-3636, © 2009
Science Publications, USA.
[2]. P. Muthuchelvi, G. S. Anandha Mala and V. Ramachandran “KBRMAS - Knowledge Based Grid Resource Management for Compromised Alternate Solution “ sai:itssa - International Transaction on System Science and Application- 2008.01.022, Vol 3, No 4, Jan 2008, ISSN 1751-1461, pp. 338-345, SIWN : Systemics and Informatics World Network Publications, UK.
[3].. Rajkumar Buyya, David Abramson and Jonathan Giddy (2000),
‘Economy driven resource management architecture for computational power grids’, In PDPTA ’00: Proceedings of the 7th International Conference on Parallel and Distributed Processing Techniques and Applications
[4]. Sim K.M (2005), Vol. 5, No. 5, pp. 22-32, ‘A survey of Bargaining models for Grid Resource allocation’, ACM SIGecom NASPs, http://doi. acm. org/10. 1145/1124566. 1124570.
[5]. Sim K.M. (2004), Vol. 20, No. 4, pp. 643-662, ‘Negotiation agents that
IJSER © 2013 http://www.ijser.org
make prudent compromises and are slightly flexible in reaching consensus’, Computational Intelligence, doi:10. 1111/j. 0824-7935.
2004. 00258.
[6]. Chee Shin Yeo and Rajkumar Buyya (2005), ‘Service Level Agreement based Allocation of Cluster Resources: Handling Penalty to Enhance Utility’, In CLUSTER ’05: Proceedings of the 7th IEEE International Conference on Cluster Computing, Boston, MA, IEEE Computer Society: Los Alamitos
[7]. Dominiak M., Kuranowski W., Gawinecki M., Ganzha M. and
Paprzycki M. (2006), ‘Utilizing agent teams in Grid resource management — preliminary considerations’, In: Proceedings of the IEEE J.V. Atanasoff Conference, IEEE CS Press, Los Alamitos, CA,46–51, http://www2. Computer.org/portal/web/ csdl/doi/10. 1109/JVA. 2006. 45-51.
[8]. Birje M.N., Manvi S.S. and Prasad B. (2006), pp. 331-334, ‘Agent based Resource Brokering and Allocation in Wireless Grids’, Proceedings of IEEE International Conference on Service Computing (SCC’06).
[9]. Somasundaram K., Radhakrishnan S. and Gomathynayagam M. (2007), Vol. 6 (5), pp. 544-547, ‘Efficient Utilization of Computing
International Journal of Scientific & Engineering Research Volume 4, Issue 2, February-2013 8
ISSN 2229-5518
Resources using Highest Response Next Scheduling in Grid’, Asian
Journal of Information Technology.
[10]. Junyan Wang, Yuebin Xu, Guanfeng Liu, Zhenkuan Pan and Yongsheng Hao (2008), ‘New Resource Discovery Mechanism with Negotiate Solution Based on Agent in Grid Environments’, IEEE The 3rd International Conference on Grid and Pervasive Computing – Workshops
[11]. Quan Liu and Yeq-ng Liao (2009), Vol. 1, pp. 556-559, ‘Grouping- Based Fine-grained Job Scheduling in Grid Computing’, IEEE First International Workshop on Education Technology and Computer Science, 2009.
[12]. Sim K.M (2010), Vol. 40, (3) pp – 245-257, ‘Grid resource negotiation: survey and new directions’, IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews. (May 2010), ISSN: 1094-6977.
[13]. Raksha Shama, Vishnu Kant Soni, Manoj Kumar Mishra and Prachet Bhuyan (2010), Vol. 64, pp. 461-466, ‘A survey of job scheduling and resource management in grid computing’, World Academy of Science, Engineering and Technology.
[14]. Manvi S.S., Birje M.N. and Prasad B. (2005), Vol. 1, No. 1,
pp. 17-27, ‘An Agent-based Resource Allocation Model for
computational Grids’, Multi Agent and Grid Systems.
[15]. Martino Di B. and Rana O.F. (2004), pp. 251-263, ‘Grid performance and resource management using mobile agents’, In: Performance Analysis and Grid Computing.
[16]. Muthuchelvi P , Anandha Mala G.S. and Ramachandran V , “ ARDNAS : Agent based Resource Discovery with Negotiated Alternate Solution”, International conference on advances in Computing ,Communication and Control.- ICAC3’09, ACM Digital library. Jan 2009. pp.239-245. ACM- ISBN-978-1-60558-351-8, DOI:
10.1145/1523103.1523154.
[17]. Muthuchelvi.P, Ramachandran .V , "ABRMAS: Agent Based Resource Management with Alternate Solution," gcc, pp. 147-153, Sixth International Conference on Grid and Cooperative Computing (GCC 2007), IEEE Xplore release 2.6. ISBN- 0- 7695-2871-6. DOI:
10.1109/GCC.2007.36.
[18].. Rajkumar Buyya, David Abramson and Srikumar Venugopal (2005), Vol. 93(3), pp. 698-714, ‘The grid economy’, Proceedings of the IEEE.
IJSER © 2013 http://www.ijser.org
International Journal of Scientific & Engineering Research Volume 4, Issue 2, February-2013 9
ISSN 2229-5518
IJSER lb)2013
http://www.ijserorq