At the snail's pace it's taking to convert inconclusive to validated, probably exacerbated by defections, I figure it's going to be long time before things return to normal, barring another server mishap.

One has to question the long term viability and commitment of a project operating on a shoestring budget, eliciting contributions from volunteers, and having no contingencies in place to avoid a repeat. Other projects have hick-ups now & then tho I'm hard pressed to point one where issues have lasted this long. Unfortunately it's sometimes a sign of forthcoming project mortality. The FLOPS this project gets is cheap computational power compared to the outlay. So hello project managers, what's your game plan? What kind of commitment can you give us? I'm happily crunching other projects for now and we would certainly appreciate some outreach from you.

looks like the hiccup the system had is almost over. My invalids are going down (slowly, but it is progress) and while waiting for validation is rising (Workunits waiting for validation 68398) it is still low. Hiccups happen, but it is nice to have Tom communicate what is going on while fixes are happening.

I have suggested we all chip in and buy some up to date hardware. Quite how Tom came up with 10 grand just for SSDs I don't know.

Or you could rent some cloud resources and attach them to the project to help speed up clearing the ready to send queue?

I am renting out some azure and aws offerings to help speed this up a bit

Hi Tom, Hi all,

this is just a calculation.

I have compared what my Raid6 will cost to Backup in a cloud and it were too expensive.
So, I had bought my 8 x8TB Hard Drives and built my own Raid6@Home

And, what should I say: It runs since 5 Years like a Charm 24/7 :-)

The only things I had to do were some managements (installing Upgrades, Cleaning the Fans).

Here is the german price from today:
https://www.idealo.de/preisvergleich/OffersOfProduct/6639979_-ironwolf-8tb-st8000vn004-seagate.html

There is no need to choose the cheapest Drive.
If you have some more traffic, just choose a helium (it's cooler) or a higher level Drive (lasts longer).

Just for the high frequented Spaces you should choose a ssd which is just acting like a buffer...

This is just my opinion, but I have also my own Raid6 with just 64TB (Useable Space ~45TB)
This is a lot of fun :-)

--
Kind Regards
Atratus

One has to question the long term viability and commitment of a project operating on a shoestring budget, eliciting contributions from volunteers, and having no contingencies in place to avoid a repeat. Other projects have hick-ups now & then tho I'm hard pressed to point one where issues have lasted this long. Unfortunately it's sometimes a sign of forthcoming project mortality. The FLOPS this project gets is cheap computational power compared to the outlay. So hello project managers, what's your game plan? What kind of commitment can you give us? I'm happily crunching other projects for now and we would certainly appreciate some outreach from you.

Because this project accepts and receives donations, it should be able to handle issues like drive failure or temporary WCG crunchers. Accepting donations seems to imply that they intend to grow the GFLOPS of the project, though, to be fair, they never say what donations will be used for.

For what purpose does this project accept donations? Does this project want or need more GFLOPS?

One has to question the long term viability and commitment of a project operating on a shoestring budget, eliciting contributions from volunteers, and having no contingencies in place to avoid a repeat. Other projects have hick-ups now & then tho I'm hard pressed to point one where issues have lasted this long. Unfortunately it's sometimes a sign of forthcoming project mortality. The FLOPS this project gets is cheap computational power compared to the outlay. So hello project managers, what's your game plan? What kind of commitment can you give us? I'm happily crunching other projects for now and we would certainly appreciate some outreach from you.

Because this project accepts and receives donations, it should be able to handle issues like drive failure or temporary WCG crunchers. Accepting donations seems to imply that they intend to grow the GFLOPS of the project, though, to be fair, they never say what donations will be used for.

For what purpose does this project accept donations? Does this project want or need more GFLOPS?

I assume the humans' pay is a big part of the money used.

And why would you assume that human pay is either a part or a big part ?

Thanks
Bill F

---

In October of 1969 I took an oath to support and defend the Constitution of the United States against all enemies, foreign and domestic;
There was no expiration date.

And why would you assume that human pay is either a part or a big part ?

Because most people don't work for free. I'm quite surprised you asked.

And why would you assume that human pay is either a part or a big part ?

Because most people don't work for free. I'm quite surprised you asked.

Well considering its a university and most of the "staff" are students at the University who actually pay to attend. I'd say they not only work for free, but they pay to work on the project in the form of tuition.

And why would you assume that human pay is either a part or a big part ?

Because most people don't work for free. I'm quite surprised you asked.

Well considering its a university and most of the "staff" are students at the University who actually pay to attend. I'd say they not only work for free, but they pay to work on the project in the form of tuition.

The professors get paid. There are running costs to everything. I can't believe you think this stuff is all free. I used to work in a university and I can tell you everything you've said is utter nonsense.

And why would you assume that human pay is either a part or a big part ?

Because most people don't work for free. I'm quite surprised you asked.

Well considering its a university and most of the "staff" are students at the University who actually pay to attend. I'd say they not only work for free, but they pay to work on the project in the form of tuition.

The professors get paid. There are running costs to everything. I can't believe you think this stuff is all free. I used to work in a university and I can tell you everything you've said is utter nonsense.

Yeah, the professors do get paid, but it's not the "MW@Home" budget and donations to the project won't go towards the professors salaries.
The grad students that help manage the project get paid with a degree.

Much like all the work we, the volunteers are doing, is also... free.

The only utter nonsense here is thinking people HAVE to get paid before they're willing to help out on anything.

Hey Tom.

Is it normal for the waiting for validation to grow? or it is a sign of an issue? Workunits waiting for validation 412415

The status page hasn't updated in a while (Task data as of 6 Apr 2022, 20:37:54 UTC) which also concerns me.

Thanks.

Yeah, the professors do get paid, but it's not the "MW@Home" budget and donations to the project won't go towards the professors salaries.
The grad students that help manage the project get paid with a degree.

Much like all the work we, the volunteers are doing, is also... free.

The only utter nonsense here is thinking people HAVE to get paid before they're willing to help out on anything.

Maybe the students are free, but others are paid for. Secretaries, IT support, etc. When I worked in a university, every department had to rent a number of rooms from the central university budget, so there's that. The server I presume is in a room. And I'm guessing for Milkyway, they have to buy telescope data.

When I was working in university it support. I know for mine for public projects you also had to pay for bandwidth that it consumed billed at 90th percentile for both inbound and outbound transfers.

Guess what I am trying to say is don't be too harsh on Tom since he is a phd student and that activity takes time away from here(such as teaching, grading, writing his thesis etc etc)

Also for a system administrator working full time it's roughly a $50k starting position

Hey Tom.

Is it normal for the waiting for validation to grow? or it is a sign of an issue? Workunits waiting for validation 412415

The status page hasn't updated in a while (Task data as of 6 Apr 2022, 20:37:54 UTC) which also concerns me.

Thanks.

I believe that is normal as people submit tasks back it adds to validation queue and if we are starting to make way in the work unit backlog. As far as I can see the validator is working just slowly

Also for a system administrator working full time it's roughly a $50k starting position

Lucky you, I got $26K, after 5 years. The UK doesn't pay workers very well. I assume you're talking US dollars?

As far as I can see the validator is working just slowly

That's an understatement. In a week maybe 3% of mine have been validated. Christmas season will be knocking before they're caught up at this rate.

I just killed some stale processes in the DB. It seems like some queries get hung up because the WU tables are quite large at the moment, and then they slow everything down because they lock certain parts of the DB while they run. Killing them every so often seems to help. I had been trying to avoid doing that to see if the server could handle things on its own at this point, but apparently it still needs the regular kicks now and then to keep functioning as per normal.

Sorry about the validation backlog and the slowness, hopefully what I just did speeds things up a bit. I'll keep killing stale tasks regularly until the validation backlog AND the WUs ready to send both decrease.

As for the pay, if you donate to milkyway, none of that goes into my (or any other milkyway personnel's) pocket. The funds are used strictly for project upgrades and development. Our salaries are funded by grants and/or teaching, not volunteer donations. We don't ask for donations because they make us richer, they just help us make the project better. Just wanted to clear that up!

As far as I can see the validator is working just slowly

That's an understatement. In a week maybe 3% of mine have been validated. Christmas season will be knocking before they're caught up at this rate.

Remember that your "validation inconclusive" also contains workunits where the other computer hasn't sent back yet.

Also for a system administrator working full time it's roughly a $50k starting position

Lucky you, I got $26K, after 5 years. The UK doesn't pay workers very well. I assume you're talking US dollars?

I live in Australia, and for us "junior system admin" is a $50-80k per year salary depending on experience. Also I was quoting Australian dollar. So USD would be $37k starting

Hey Tom:

I guess now that the Separation queue ready to send is almost 0, you can turn that work generator back on