Hi Everyone,

I just put some new Separation runs up on the server and took down the old ones. You may still see new workunits from old runs in your queues for a few days as those runs finish validating.

The names of the new runs are:
de_modfit_80_bundle4_4s_south4s_bgset
de_modfit_81_bundle4_4s_south4s_bgset
de_modfit_82_bundle4_4s_south4s_bgset
de_modfit_83_bundle4_4s_south4s_bgset
de_modfit_84_bundle4_4s_south4s_bgset
de_modfit_85_bundle4_4s_south4s_bgset
de_modfit_86_bundle4_4s_south4s_bgset

An error processing a flag in the parameter files has been fixed and updated runs have been released. These runs are confirmed to be returning results as of 10 PM on 7/24. I apologize for the inconvenience.

The names of the new runs are:
de_modfit_80_bundle4_4s_south4s_bgset_2
de_modfit_81_bundle4_4s_south4s_bgset_2
de_modfit_82_bundle4_4s_south4s_bgset_2
de_modfit_83_bundle4_4s_south4s_bgset_2
de_modfit_84_bundle4_4s_south4s_bgset_2
de_modfit_85_bundle4_4s_south4s_bgset_2
de_modfit_86_bundle4_4s_south4s_bgset_2

As these runs optimize we may see increased invalidated returns with the stripe 84 and 85 runs. This is a known issue that I believe has something to do with a data cut made in those stripes. If/when this starts happening and the stripes are sufficiently optimized, I will take them down so you all don't have to worry about crunching invalidated WUs.

If you have any questions/comments/concerns, please feel free to post them here. Thanks for all your support!

Best,
Tom

Hi Tom,
I just got a bunch of errors with this new runs. This happened on all 3 of my machines which were fine before this

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<message>
Incorrect function.
(0x1) - exit code 1 (0x1)</message>
<stderr_txt>
<search_application> milkyway_separation 1.46 Windows x86 double OpenCL </search_application>
BOINC GPU type suggests using OpenCL vendor 'Advanced Micro Devices, Inc.'
Error loading Lua script 'astronomy_parameters.txt': [string "number_parameters: 4..."]:1: '<name>' expected near '4'
Switching to Parameter File 'astronomy_parameters.txt'
<number_WUs> 4 </number_WUs>
<number_params_per_WU> 25 </number_params_per_WU>
Number of streams does not match
16:49:44 (8604): called boinc_finish(1)

</stderr_txt>
]]

Edit: I'll stop getting work for now because my errors went up faster than my valids :(

Thanks for bringing that to my attention, none of the runs are returning data. I'll try to fix that as soon as I can.

Tom

Same problem, got over 500 error'ed out tasks with message "number of streams do not match"

Please post when tasks are ready as I have to stop processing.

I have updated runs and taken down the bugged ones, waiting to make sure that they are returning results. I'll update the main post once I make sure that these runs are functioning correctly.

If you have errors with runs ending in "...bgset_2", then I would appreciate seeing your error messages.

Sorry for the inconvenience!

Came here because all my tasks started failing. Glad its not just me i guess...

This is what I got:
<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
Incorrect function.
(0x1) - exit code 1 (0x1)</message>
<stderr_txt>
<search_application> milkyway_separation 1.46 Windows x86 double OpenCL </search_application>
Reading preferences ended prematurely
BOINC GPU type suggests using OpenCL vendor 'Advanced Micro Devices, Inc.'
Error loading Lua script 'astronomy_parameters.txt': [string "number_parameters: 4..."]:1: '<name>' expected near '4'
Switching to Parameter File 'astronomy_parameters.txt'
<number_WUs> 4 </number_WUs>
<number_params_per_WU> 25 </number_params_per_WU>
Number of streams does not match
06:50:25 (6780): called boinc_finish(1)

</stderr_txt>
]]>

Is this from a "...bgset" run, or a "...bgset_2" run? The new "...bgset_2" runs ran fine on my client. This looks identical to the last error for the previous bugged runs. Thank you for the input, though!

Tom

Hi Tom,
I enabled 2 of my machine just now. Still got some errors, i guess the old runs.
The bgset_2 looks like this

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
<search_application> milkyway_separation 1.46 Windows x86 double OpenCL </search_application>
BOINC GPU type suggests using OpenCL vendor 'Advanced Micro Devices, Inc.'
Error loading Lua script 'astronomy_parameters.txt': [string "number_parameters: 4..."]:1: '<name>' expected near '4'
Switching to Parameter File 'astronomy_parameters.txt'
<number_WUs> 4 </number_WUs>
<number_params_per_WU> 26 </number_params_per_WU>
Using SSE4.1 path
Found 2 platforms
Platform 0 information:
Name: AMD Accelerated Parallel Processing
Version: OpenCL 2.0 AMD-APP (2442.12)
Vendor: Advanced Micro Devices, Inc.
Extensions: cl_khr_icd cl_khr_d3d10_sharing cl_khr_d3d11_sharing cl_khr_dx9_media_sharing cl_amd_event_callback cl_amd_offline_devices
Profile: FULL_PROFILE
Platform 1 information:
Name: Intel(R) OpenCL
Version: OpenCL 2.1
Vendor: Intel(R) Corporation
Extensions: cl_intel_dx9_media_sharing cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_d3d11_sharing cl_khr_depth_images cl_khr_dx9_media_sharing cl_khr_fp64 cl_khr_gl_sharing cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_image2d_from_buffer cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_spir
Profile: FULL_PROFILE
Using device 1 on platform 0
Found 3 CL devices
Device 'Tahiti' (Advanced Micro Devices, Inc.:0x1002) (CL_DEVICE_TYPE_GPU)
Board: AMD Radeon R9 200 Series
Driver version: 2442.12
Version: OpenCL 1.2 AMD-APP (2442.12)
Compute capability: 0.0
Max compute units: 32
Clock frequency: 1100 Mhz
Global mem size: 3221225472
Local mem size: 32768
Max const buf size: 65536
Double extension: cl_khr_fp64
Estimated AMD GPU GFLOP/s: 4506 SP GFLOP/s, 1126 DP FLOP/s
Using a target frequency of 60.0
Using a block size of 8192 with 68 blocks/chunk
Using clWaitForEvents() for polling (mode -1)
Range: { nu_steps = 320, mu_steps = 800, r_steps = 700 }
Iteration area: 560000
Chunk estimate: 1
Num chunks: 2
Chunk size: 557056
Added area: 554112
Effective area: 1114112
Initial wait: 16 ms
Integration time: 13.201212 s. Average time per iteration = 41.253788 ms
Integral 0 time = 13.681410 s
Running likelihood with 38095 stars
Likelihood time = 0.920602 s
<background_integral> 0.000155305800785 </background_integral>
<stream_integral> 4.181364298317329 1.584709335463284 130.381645952612530 48.033929790624995 </stream_integral>
<background_likelihood> -10.909361772835503 </background_likelihood>
<stream_only_likelihood> -8.764722699084267 -153.382575093746400 -3.053178863333347 -7.281001906390212 </stream_only_likelihood>
<search_likelihood> -3.045379847005349 </search_likelihood>
Using SSE4.1 path
Found 2 platforms
Platform 0 information:
Name: AMD Accelerated Parallel Processing
Version: OpenCL 2.0 AMD-APP (2442.12)
Vendor: Advanced Micro Devices, Inc.
Extensions: cl_khr_icd cl_khr_d3d10_sharing cl_khr_d3d11_sharing cl_khr_dx9_media_sharing cl_amd_event_callback cl_amd_offline_devices
Profile: FULL_PROFILE
Platform 1 information:
Name: Intel(R) OpenCL
Version: OpenCL 2.1
Vendor: Intel(R) Corporation
Extensions: cl_intel_dx9_media_sharing cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_d3d11_sharing cl_khr_depth_images cl_khr_dx9_media_sharing cl_khr_fp64 cl_khr_gl_sharing cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_image2d_from_buffer cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_spir
Profile: FULL_PROFILE
Using device 1 on platform 0
Found 3 CL devices
Device 'Tahiti' (Advanced Micro Devices, Inc.:0x1002) (CL_DEVICE_TYPE_GPU)
Board: AMD Radeon R9 200 Series
Driver version: 2442.12
Version: OpenCL 1.2 AMD-APP (2442.12)
Compute capability: 0.0
Max compute units: 32
Clock frequency: 1100 Mhz
Global mem size: 3221225472
Local mem size: 32768
Max const buf size: 65536
Double extension: cl_khr_fp64
Estimated AMD GPU GFLOP/s: 4506 SP GFLOP/s, 1126 DP FLOP/s
Using a target frequency of 60.0
Using a block size of 8192 with 68 blocks/chunk
Using clWaitForEvents() for polling (mode -1)
Range: { nu_steps = 320, mu_steps = 800, r_steps = 700 }
Iteration area: 560000
Chunk estimate: 1
Num chunks: 2
Chunk size: 557056
Added area: 554112
Effective area: 1114112
Initial wait: 16 ms
Integration time: 13.226758 s. Average time per iteration = 41.333619 ms
Integral 0 time = 13.749233 s
Running likelihood with 38095 stars
Likelihood time = 0.873341 s
<background_integral1> 0.000136224675998 </background_integral1>
<stream_integral1> 152.671899805996900 145.555865287139110 183.784669581012310 4.906343751930823 </stream_integral1>
<background_likelihood1> -9.784610545126764 </background_likelihood1>
<stream_only_likelihood1> -17.027740007600620 -2.898074963499768 -13.385826642313273 -136.031850343614370 </stream_only_likelihood1>
<search_likelihood1> -2.898073595535414 </search_likelihood1>
Using SSE4.1 path
Found 2 platforms
Platform 0 information:
Name: AMD Accelerated Parallel Processing
Version: OpenCL 2.0 AMD-APP (2442.12)
Vendor: Advanced Micro Devices, Inc.
Extensions: cl_khr_icd cl_khr_d3d10_sharing cl_khr_d3d11_sharing cl_khr_dx9_media_sharing cl_amd_event_callback cl_amd_offline_devices
Profile: FULL_PROFILE
Platform 1 information:
Name: Intel(R) OpenCL
Version: OpenCL 2.1
Vendor: Intel(R) Corporation
Extensions: cl_intel_dx9_media_sharing cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_d3d11_sharing cl_khr_depth_images cl_khr_dx9_media_sharing cl_khr_fp64 cl_khr_gl_sharing cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_image2d_from_buffer cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_spir
Profile: FULL_PROFILE
Using device 1 on platform 0
Found 3 CL devices
Device 'Tahiti' (Advanced Micro Devices, Inc.:0x1002) (CL_DEVICE_TYPE_GPU)
Board: AMD Radeon R9 200 Series
Driver version: 2442.12
Version: OpenCL 1.2 AMD-APP (2442.12)
Compute capability: 0.0
Max compute units: 32
Clock frequency: 1100 Mhz
Global mem size: 3221225472
Local mem size: 32768
Max const buf size: 65536
Double extension: cl_khr_fp64
Estimated AMD GPU GFLOP/s: 4506 SP GFLOP/s, 1126 DP FLOP/s
Using a target frequency of 60.0
Using a block size of 8192 with 68 blocks/chunk
Using clWaitForEvents() for polling (mode -1)
Range: { nu_steps = 320, mu_steps = 800, r_steps = 700 }
Iteration area: 560000
Chunk estimate: 1
Num chunks: 2
Chunk size: 557056
Added area: 554112
Effective area: 1114112
Initial wait: 16 ms
Integration time: 13.202470 s. Average time per iteration = 41.257717 ms
Integral 0 time = 13.680797 s
Running likelihood with 38095 stars
Likelihood time = 0.857028 s
<background_integral2> 0.000165617672821 </background_integral2>
<stream_integral2> 160.847266587313300 3.968876879409631 176.308152947632440 129.096313328206120 </stream_integral2>
<background_likelihood2> -8.366224710913647 </background_likelihood2>
<stream_only_likelihood2> -12.668971218100111 -4.873254839594230 -6.613812086197211 -9.336805624681906 </stream_only_likelihood2>
<search_likelihood2> -4.346214889695199 </search_likelihood2>
Using SSE4.1 path
Found 2 platforms
Platform 0 information:
Name: AMD Accelerated Parallel Processing
Version: OpenCL 2.0 AMD-APP (2442.12)
Vendor: Advanced Micro Devices, Inc.
Extensions: cl_khr_icd cl_khr_d3d10_sharing cl_khr_d3d11_sharing cl_khr_dx9_media_sharing cl_amd_event_callback cl_amd_offline_devices
Profile: FULL_PROFILE
Platform 1 information:
Name: Intel(R) OpenCL
Version: OpenCL 2.1
Vendor: Intel(R) Corporation
Extensions: cl_intel_dx9_media_sharing cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_d3d11_sharing cl_khr_depth_images cl_khr_dx9_media_sharing cl_khr_fp64 cl_khr_gl_sharing cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_image2d_from_buffer cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_spir
Profile: FULL_PROFILE
Using device 1 on platform 0
Found 3 CL devices
Device 'Tahiti' (Advanced Micro Devices, Inc.:0x1002) (CL_DEVICE_TYPE_GPU)
Board: AMD Radeon R9 200 Series
Driver version: 2442.12
Version: OpenCL 1.2 AMD-APP (2442.12)
Compute capability: 0.0
Max compute units: 32
Clock frequency: 1100 Mhz
Global mem size: 3221225472
Local mem size: 32768
Max const buf size: 65536
Double extension: cl_khr_fp64
Estimated AMD GPU GFLOP/s: 4506 SP GFLOP/s, 1126 DP FLOP/s
Using a target frequency of 60.0
Using a block size of 8192 with 68 blocks/chunk
Using clWaitForEvents() for polling (mode -1)
Range: { nu_steps = 320, mu_steps = 800, r_steps = 700 }
Iteration area: 560000
Chunk estimate: 1
Num chunks: 2
Chunk size: 557056
Added area: 554112
Effective area: 1114112
Initial wait: 16 ms
Integration time: 11.259764 s. Average time per iteration = 35.186763 ms
Integral 0 time = 11.575826 s
Running likelihood with 38095 stars
Likelihood time = 0.967706 s
<background_integral3> 0.000086545095616 </background_integral3>
<stream_integral3> 0.135624606604846 66.422776529197293 173.534953341131880 113.367127592535130 </stream_integral3>
<background_likelihood3> -8.873801050948440 </background_likelihood3>
<stream_only_likelihood3> -214.679271661823320 -6.563516836358728 -2.915561601617464 -16.132769365806681 </stream_only_likelihood3>
<search_likelihood3> -2.915368518622770 </search_likelihood3>
00:00:22 (8948): called boinc_finish(0)

</stderr_txt>
]]>


I'll monitor to see if any error with the bgset_2 ... but looks good so far. Thanks for the quick fix!

Edit: confirmed, the errors were the other runs de_modfit_83_bundle4_4s_south4s_bgset_

Is this from a "...bgset" run, or a "...bgset_2" run? The new "...bgset_2" runs ran fine on my client. This looks identical to the last error for the previous bugged runs. Thank you for the input, though!

Tom

Oh right yes my mistake, that was from a bgset run, not a bgset_2 run. Looks like bgset_2 is running all good on my end.

Hello,
Also at me about 50% in error on two host (772323 and 796366)
ATI RX580
But it is not always depending of WU. Some bigset_2 are ok, some not.
Bigset_1 and bigset the same problem
Best regards

---

Hi Marsinph,

Can you describe the error, or post an error log here so I can be more helpful?

Tom

Also seeing ~ 30% errors on all hosts here: 2 Titan Vs, radeon VII, 295x2 and Radeon 290.

This is typical of thousands.

Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
Incorrect function.
(0x1) - exit code 1 (0x1)</message>
<stderr_txt>
<search_application> milkyway_separation 1.46 Windows x86 double OpenCL </search_application>
Reading preferences ended prematurely
BOINC GPU type suggests using OpenCL vendor 'Advanced Micro Devices, Inc.'
Error loading Lua script 'astronomy_parameters.txt': [string "number_parameters: 4..."]:1: '<name>' expected near '4'
Switching to Parameter File 'astronomy_parameters.txt'
<number_WUs> 4 </number_WUs>
<number_params_per_WU> 25 </number_params_per_WU>
Number of streams does not match
22:26:36 (1368): called boinc_finish(1)

</stderr_txt>
]]>

This is an example with full information. I have had about seven thousand errors.

Name de_modfit_85_bundle4_4s_south4s_bgset_1561047003_12568144_2
Workunit 1784531783
Created 26 Jul 2019, 1:35:00 UTC
Sent 26 Jul 2019, 2:01:22 UTC
Report deadline 7 Aug 2019, 2:01:22 UTC
Received 26 Jul 2019, 2:28:32 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 808022
Run time 1 sec
CPU time
Validate state Invalid
Credit 0.00
Device peak FLOPS 823.78 GFLOPS
Application version Milkyway@home Separation v1.46 (opencl_ati_101)
windows_x86_64
Peak disk usage 0.01 MB
Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
Incorrect function.
(0x1) - exit code 1 (0x1)</message>
<stderr_txt>
<search_application> milkyway_separation 1.46 Windows x86 double OpenCL </search_application>
Reading preferences ended prematurely
BOINC GPU type suggests using OpenCL vendor 'Advanced Micro Devices, Inc.'
Error loading Lua script 'astronomy_parameters.txt': [string "number_parameters: 4..."]:1: '<name>' expected near '4'
Switching to Parameter File 'astronomy_parameters.txt'
<number_WUs> 4 </number_WUs>
<number_params_per_WU> 25 </number_params_per_WU>
Number of streams does not match
22:26:36 (1368): called boinc_finish(1)

</stderr_txt>
]]>

This looks like it is still from the old bugged runs. It is a "...bgset" run, and the new runs will have 26 parameters, not 25. Can you clear your queue and see if that helps?

Tom

I was away from home when you asked if I could clear the queue. The queue was empty when I returned and I updated to get new work. The new tasks are running without errors.

This looks like it is still from the old bugged runs. It is a "...bgset" run, and the new runs will have 26 parameters, not 25. Can you clear your queue and see if that helps?

Tom

I've clear my queue several times and still get errors across hosts... then it puts in a 24 hour update delay. Hit "Update" and it downloads a few hundred work units, which proceed to error at about 50% of those that don't.

I have had about three thousand work unit errors during a four hour period ending with the task quoted. A quick review found the errors to be mostly on de_modfit_85_bundle4_4s_south4s_bgset_2 and de_modfit_86_bundle4_4s_south4s_bgset_2. This is a typical error message:

Task 278472433
Name de_modfit_80_bundle4_4s_south4s_bgset_1561047003_12558111_2
Workunit 1784520616
Created 28 Jul 2019, 9:54:26 UTC
Sent 28 Jul 2019, 10:17:37 UTC
Report deadline 9 Aug 2019, 10:17:37 UTC
Received 28 Jul 2019, 11:14:41 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 808022
Run time 2 sec
CPU time
Validate state Invalid
Credit 0.00
Device peak FLOPS 823.78 GFLOPS
Application version Milkyway@home Separation v1.46 (opencl_ati_101)
windows_x86_64
Peak working set size 10.63 MB
Peak swap size 6.41 MB
Peak disk usage 0.01 MB
Stderr output

<core_client_version>7.14.2</core_client_version>
<![CDATA[
<message>
Incorrect function.
(0x1) - exit code 1 (0x1)</message>
<stderr_txt>
<search_application> milkyway_separation 1.46 Windows x86 double OpenCL </search_application>
Reading preferences ended prematurely
BOINC GPU type suggests using OpenCL vendor 'Advanced Micro Devices, Inc.'
Error loading Lua script 'astronomy_parameters.txt': [string "number_parameters: 4..."]:1: '<name>' expected near '4'
Switching to Parameter File 'astronomy_parameters.txt'
<number_WUs> 4 </number_WUs>
<number_params_per_WU> 25 </number_params_per_WU>
Number of streams does not match
07:13:15 (2824): called boinc_finish(1)

</stderr_txt>
]]>

Vester those aren't "bgset_2" work units. Those are the bugged "bgset" work units.

"bgset_2" work units have the "_2" right next to the "bgset", like this:

de_modfit_84_bundle4_4s_south4s_bgset_2_1564052102_378831_1