Welcome to MilkyWay@home

Computation parameters

Message boards : Number crunching : Computation parameters
Message board moderation

To post messages, you must log in.

AuthorMessage
hoarfrost

Send message
Joined: 4 Sep 09
Posts: 20
Credit: 187,688,252
RAC: 0
Message 31254 - Posted: 20 Sep 2009, 18:18:52 UTC

Hello all! :-)

If I understood right, computation parameters for each task put in one very small (about 200-800 bytes) file that BOINC must download from server. And output results must be written in very small file, that BOINC upload to server.

If this parameters write into WU-info, that BOINC get in task request and write into client_state.xml as command-line arguments (for example), and computation result also write into "fields" of "task-report request" - may be server loading decrease?

P.S. Excuse me please if my English is not right. :-)
ID: 31254 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Paul D. Buck

Send message
Joined: 12 Apr 08
Posts: 621
Credit: 161,934,067
RAC: 0
Message 31255 - Posted: 20 Sep 2009, 18:45:39 UTC

Generally speaking the files issued by projects that we call tasks, and the result files are both fairly small. Much of the work handling these files in any case is off-loaded to a secondary server (assuming multiple servers) called the upload/download handler.

So, in a theoretical minimalist system we would have

a) Database Server
b) File Server
c) Scheduler

Unfortunately, many small projects of which MW is one have only one or two servers that do all the work.

More unfortunately is the fact that at this time the tasks can be completed in such short time periods. GPU Grids tasks are, even on fast GPUs, about 6 to 10 hours in length. Collatz tasks are 15 minutes or more, and so on ... My 4870 GPU does MW tasks in under a minute each ... that means lots of calls to the server. At least once every 20 minutes or so ... 3 times an hour, 72 times a day and that is only to get the tasks ... I am uploading a result once per minute or as many as 1440 times a day in addition to that just to upload results ...

It is when you multiply that times all the other people doing the same thing that it gets to be a problem.

The "sure cure" would be if we could package the work so that we would be doing 10 times as much and taking 10 times longer to do so ... hopefully with 10 times the credit of course ... that would cut the load on the server by at least that much ... but we would get the same work done ...

Not sure If I answered the question or not ... and some might quibble about the numbers and my conclusion ... but ... for a fast answer ...
ID: 31255 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
hoarfrost

Send message
Joined: 4 Sep 09
Posts: 20
Credit: 187,688,252
RAC: 0
Message 31256 - Posted: 20 Sep 2009, 19:05:20 UTC - in response to Message 31255.  

More unfortunately is the fact that at this time the tasks can be completed in such short time periods. GPU Grids tasks are, even on fast GPUs, about 6 to 10 hours in length. Collatz tasks are 15 minutes or more, and so on ... My 4870 GPU does MW tasks in under a minute each ... that means lots of calls to the server. At least once every 20 minutes or so ... 3 times an hour, 72 times a day and that is only to get the tasks ... I am uploading a result once per minute or as many as 1440 times a day in addition to that just to upload results ...

It is when you multiply that times all the other people doing the same thing that it gets to be a problem.


MilkyWay@Home project RAC near the 50 000 000. WU "price" ~ 50 CS. MW need operate about 1 000 000 WU's per one day. 1 WU = 2 files = 1 input + 1 output.
MW@H must operate about (2 000 000 file / day) / (86 400 sec / day) ~ 23 file / second.
ID: 31256 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Computation parameters

©2024 Astroinformatics Group