bulk data load errors in release 32

Post Reply
ssteiner
Member
Posts: 58
Joined: 05 Feb 2009 09:20

bulk data load errors in release 32

Post by ssteiner » 12 May 2019 13:33

I've decided to give O2G another go in my provisioning solution. But, it fails right there at the data load (I cache various data so the GUI won't have to fetch it from the PBX every time somebody accesses the GUI). Data caches ist stuff like Entities, Costcenter, Device Templates, etc.

So I extract the list of objects, then I load every object returned in the list of objectIds (I can't use the attributes tag since that was broken from v029 to v032). So, loading my 2000 costcenters results in a majority of successful loads, and a couple dozen of 503s

Code: Select all

HTTP/1.1 503 Service Unavailable
Date: Sun, 12 May 2019 17:18:16 GMT
Server: Apache/2.4.23
Content-Length: 227
Connection: close
Content-Type: application/json

{"httpStatus":"SERVICE_UNAVAILABLE","code":1000,"helpMessage":"The server is not yet available. Please retry later if the initialization is not finished.","type":"NOT_READY","innerMessage":"Resource limitation","canRetry":true}
Subsequent loads work, there doesn't seem to be any regularly, except that in every attempt I make, I have at least a handful of these. When I originally wrote the code, with the pre-release and 1.0 release, things worked fine, so something was broken in subsequent releases.

yblanchard
Member
Posts: 25
Joined: 03 Dec 2018 06:32

Re: bulk data load errors in release 32

Post by yblanchard » 14 May 2019 10:06

we didn't see anything brought by last version which could explain this behahavior.

However, it will be interesting to know how the requests are thrown: are they in sequential mode or in parallel mode ?

We can already say that OXE CMIS simultaneous client number is limited ( probably maximum 4 parallel requests)

Our lab test (getting all the Subscriber data) show that using 4 parrallel requests leads to the better result, and since 8, some errors appear.

Plassard
Member
Posts: 4
Joined: 27 Mar 2019 03:30

Re: bulk data load errors in release 32

Post by Plassard » 15 May 2019 06:17

This issue will be fixed with CRO2G-68 in version 33
Concurrent request number will be limited at O2G level

ssteiner
Member
Posts: 58
Joined: 05 Feb 2009 09:20

Re: bulk data load errors in release 32

Post by ssteiner » 23 May 2019 05:18

The requests are fired parallel. There's a single thread starting the extraction for every item and these use async (Task Parallel Framework in .NET). Nothing has changed in that code though - once listing items with selected attributes work again, I'll rewrite as much as possible to use the attribute selection approach which means a single request getting large datasets instead of making a large number of requests.

Is there a limit to the number of attributes that can be extracted? Worst case there's perhaps a dozen attributes i need...

When is release 33 due?

Post Reply

Return to “O2G”