Making Parallel Requests using Typhoeus

Making Parallel Requests

# Generally, you should be running requests through hydra. Here is how that looks
hydra = Typhoeus::Hydra.new

first_request = Typhoeus::Request.new("http://localhost:3000/posts/1.json")
first_request.on_complete do |response|
  post = JSON.parse(response.body)
  third_request = Typhoeus::Request.new(post.links.first) # get the first url in the post
  third_request.on_complete do |response|
    # do something with that
  end
  hydra.queue third_request
  return post
end
second_request = Typhoeus::Request.new("http://localhost:3000/users/1.json")
second_request.on_complete do |response|
  JSON.parse(response.body)
end
hydra.queue first_request
hydra.queue second_request
hydra.run # this is a blocking call that returns once all requests are complete

first_request.handled_response # the value returned from the on_complete block
second_request.handled_response # the value returned from the on_complete block (parsed JSON)

The execution of that code goes something like this. The first and second requests are built and queued. When hydra is run the first and second requests run in parallel. When the first request completes, the third request is then built and queued up. The moment it is queued Hydra starts executing it. Meanwhile the second request would continue to run (or it could have completed before the first). Once the third request is done, hydra.run returns.