通信学报

• 论文II • 上一篇    下一篇

基于节点识别的慢任务调度算法

崔云飞,李新明,李 艺,刘 东   

  1. 1. 北京航天飞行控制中心,北京 100094;2. 装备学院 复杂电子系统仿真重点实验室,北京 101416
  • 出版日期:2014-07-25 发布日期:2014-07-15
  • 基金资助:
    国家自然科学基金资助项目(60904082);国家重大科技专项基金项目(2012ZX01045003-001)

Slow task scheduling algorithm based on node identification

  • Online:2014-07-25 Published:2014-07-15

摘要: 为了降低大数据处理集群在执行任务过程中的慢任务对作业执行效率的影响,提出了一种识别慢任务、备份慢任务、减少慢任务相结合的调度算法——TQST算法。首先,通过判断节点能力和任务执行时间,建立慢节点、非常慢节点和慢任务队列;其次,根据预判备份执行价值确定如何启动慢任务的备份任务,提高了备份执行的作用;然后,在节点识别的基础上,规避为非常慢节点分配任务,从根本上减少慢任务的产生,提高作业执行效率。实验结果表明,TQST算法在作业响应时间等方面优于已有的慢任务调度算法。

Abstract: In order to reduce the influence of the slow task, produced in big data processing, a scheduling algorithm (TQST) combining recognition, speculation and seduction of slow task was proposed. First of all, through the judgment of node ability and task execution time, slow node queue, very slow node queue and slow task queue were established. Secondly, according to the anticipation speculative execution value to decide how to start speculative task. Then, in the basis of node identification, avoid distributing tasks to very slow node, radically reduce slow task production, improve job execution efficiency. The experimental results show that TQST algorithm previous existing slow task scheduling algorithm in term of the job response time.

No Suggested Reading articles found!