Hadoop Yarn ATS Rest APIs

For Yarn application, to fetch application’s data we can use Rest APIs on ATS below are some reference links:

https://community.hortonworks.com/content/supportkb/221899/how-to-export-information-from-tez-view.html

https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/TimelineServer.html#Timeline_Server_REST_API_V1

We can use RM REST APIs to get some application related data. Some of the Examples are below:

a) Failed apps for the specific time period

GET “http://Resource-Manager-Address:8088/ws/v1/cluster/apps?limit=10&startedTimeBegin={time in epoch}&startedTimeEnd={time in epoch}&states=FAILED”

b) Query To get all the apps having states as FINISHED,KILLED by the specific user for specific time period

GET “http://Resource-Manager-Address:8088/ws/v1/cluster/apps?limit=20&states=FINISHED,KILLED&user=<user-id>&startedTimeBegin={time in epoch}&startedTimeEnd={time in epoch}”

c)To get failed jobs for the specific user and for specific time period

GET “http://Resource-Manager-Address:8088/ws/v1/cluster/apps?limit=1&startedTimeBegin={time in epoch}&startedTimeEnd={time in epoch}&states=FAILED&user=<user-id>”

d) To get finished jobs for the specific user

GET “http://Resource-Manager-Address:8088/ws/v1/cluster/apps?limit=1&states=FINISHED&user=<user-id>"

e) To get finished jobs based on the application type

GET “http://Resource-Manager-Address:8088/ws/v1/cluster/apps?limit=1&states=FINISHED&applicationTypes=tez"

f)Get spark application type jobs

GET “http://Resource-Manager-Address:8088/ws/v1/cluster/apps?limit=1&states=FINISHED&applicationTypes=spark"