Picture this:
You find yourself with a big list of resource identifiers
You want to make get information for each id via a http request
You want to batch the resource identifiers rather than making individual http calls
You want to make multiple parallel requests for these batches
You don’t want to manage multiple thread access to a single response collection
You have come to the right place!
There are multiple approaches I can think of, but here’s one that may fit your use case:
Get Method:
|
The code in the method itself is not that important!
what you should take from it is that it’s an ordinary async/await method that takes a list of Ids and returns the results for same.
To Parallelizm and beyond
|
Let’s unravel what’s happening above.
Firstly we are using .net6 where we have the Chunk method from the framework (gone are the days of writing this by hand thankfully!),
the chunk method basically takes the ids list and breaks it into a list of smaller lists of size ‘batchSize’ or smaller.
e.g.
If we chunked [1,2,3,4,5,6,7,8,9,10,11] with a batch size of 2, then we’d end up with
[[1,2],[3,4],[5,6],[7,8],[9,10],[11]]
|
Secondly we pass these smaller arrays to the GetIds call, by using a Linq Select expression.
We await the result of all these Selected tasks via Task.WhenAll
Lastly to combine all the individual batched responses we use the Linq SelectMany expression.
I like this approach as it is terse, concise and propagates exceptions without wrapping them in Aggregates.
|