Please or Register to create posts and topics.

Anyone Know How the OPUS Queue Works?

#11 · October 25, 2024, 12:42 pm

Quote from jalbrz on October 25, 2024, 12:42 pm
First thing to note is that the OPUS servers utilize 2 separate queues.

Second thing to note is that no one is manually filtering OPUS uploads based on email addresses of users who have frustrated them. The OPUS queuing, processing, and delivering is all automated, there is only human intervention when things go wrong.

There is the Regular queue that the average user upload goes into, like when you submit 3-5 files in a morning of post-processing. These are processed in the order they are received. Note that whenever the OPUS servers have to translate (like when uploading a .DAT instead of RINEX) or decimate (like uploading a 5 s logging rate file... which is cut down to 30 s logging rate anyway) an uploaded file, that adds not just to your processing time but also to everyone else's queueing time. I believe this is a part of the reason that future versions of OPUS will only accept RINEX uploads.

Along with the Regular queue, there is the Overflow queue that handles the "power user" uploads. These are also processed in the order they are received, but this queue can end up much longer than the Regular. How do the OPUS servers decide what is a "power user" to send to the Overflow? There is a 50 upload threshold per day; once an you exceed the magic number of 50 uploads, all of your solutions are migrated from the Regular to the Overflow queue. I do not know if the "day" is defined by UTC midnight-to-midnight, or a previous rolling 24 hour period. The Overflow queue has at times been upwards of 30k files. And just like in the Regular queue, if Overflow users are uploading files that OPUS servers need to translate or decimate before processing... well then, that can get pretty backed up.

Unfortunately, I don't have an answer to which queue the OPUS Today graphic is illustrating (i.e. Regular, Overflow, both?) but I will ask for details.

Did you go over the 50 solution limit on the day you had delays? That's my best guess, but I was wrong once before.

Hope this helps clarify.

First thing to note is that the OPUS servers utilize 2 separate queues.

Second thing to note is that no one is manually filtering OPUS uploads based on email addresses of users who have frustrated them. The OPUS queuing, processing, and delivering is all automated, there is only human intervention when things go wrong.

There is the Regular queue that the average user upload goes into, like when you submit 3-5 files in a morning of post-processing. These are processed in the order they are received. Note that whenever the OPUS servers have to translate (like when uploading a .DAT instead of RINEX) or decimate (like uploading a 5 s logging rate file... which is cut down to 30 s logging rate anyway) an uploaded file, that adds not just to your processing time but also to everyone else's queueing time. I believe this is a part of the reason that future versions of OPUS will only accept RINEX uploads.

Along with the Regular queue, there is the Overflow queue that handles the "power user" uploads. These are also processed in the order they are received, but this queue can end up much longer than the Regular. How do the OPUS servers decide what is a "power user" to send to the Overflow? There is a 50 upload threshold per day; once an you exceed the magic number of 50 uploads, all of your solutions are migrated from the Regular to the Overflow queue. I do not know if the "day" is defined by UTC midnight-to-midnight, or a previous rolling 24 hour period. The Overflow queue has at times been upwards of 30k files. And just like in the Regular queue, if Overflow users are uploading files that OPUS servers need to translate or decimate before processing... well then, that can get pretty backed up.

Unfortunately, I don't have an answer to which queue the OPUS Today graphic is illustrating (i.e. Regular, Overflow, both?) but I will ask for details.

Did you go over the 50 solution limit on the day you had delays? That's my best guess, but I was wrong once before.

Hope this helps clarify.

#12 · October 25, 2024, 1:44 pm

Thanks for the explanation! I didn't go over 50 that day, I don't think, so I expect I was in the regular queue. I just happened to be submitting when the backup was substantial enough that for a couple of days there was a banner noting that processing times would be much longer than usual. That banner isn't there now, so whatever unusual activity had been occurring appears to have abated.

Now, if I can just figure out why OPUS Projects won't respond to my adjustment submittals...