Deployment Tasks not visible in Snap Deploy Console

Background info: Snap Deploy is running in Windows 10 LTSC on a bare-metal server on my LAN. We typically run deployments via Console on the same machine with just one Windows login. Everything working perfectly for up to a couple weeks. Then I log in one day to see this:

Every other part of Snap Deploy seems to work just fine. I can still manage the PXE server and I can still do deployments via Standalone Deployment, and the Machines screen reports machine status accurately. Tasks even still appear in user-initiated deployment mode, but that solution will not work for us because Snap Deploy's built-in (non-WinPE) PXE environment and bootable media do not include drivers for the Lenovo P15 Gen 2, which makes up most of our inventory. As long as we can't initiate tasks from Console, we have to use Standalone for everything, which is a significant slowdown.

Fix attempt #1: I tried refreshing a few times; no change.

Fix attempt #2: Recreate task. Then I see this:

Fix attempt #3: Reboot. No change.

Fix attempt #4: Uninstall all Acronis software via Windows Settings -> Apps -> Apps & Features, reboot, and reinstall. No change.

Fix attempt #5: Uninstall all Acronis software via Snap Deploy installation tool, reboot, and reinstall. No change.

Fix attempt #6: Use Snap Deploy installer to "repair installation." The tool reported no issues found, and the issue was not resolved.

Fix attempt #7: Contact Acronis support. At first they recommended a sort of universal Acronis repair tool, but Snap Deploy is not among the products for which that tool was designed, so it didn't work for me. They were unable to determine the cause of the issue, but took a lot of logs and information from my machine so they could investigate. I have not heard back, but our support period has expired since then.

Fix attempt #8: Reinstall Windows and re-configure from scratch. Once I had all my machines, tasks, and licenses configured and confirmed that my tasks were visible, I took an image of this imaging server with another imaging server so I could restore a known working state if this issue were to happen again. This time, I also disabled automatic Windows updates via Group Policy. Like before, the setup worked perfectly for about two weeks before my tasks disappeared again.

Fix attempt #9: Reimaged with the backup image I created before, confirmed working as expected, ran Windows update, and continued working as expected. (Windows update did not cause the issue.) Captured a new image, left automatic updates disabled in Group Policy just to be on the safe side. Continued working perfectly for about two weeks before my tasks disappeared again.

Unintentional fix attempt #10: Another user rebooted the machine, and the tasks came back and everything was working fine again. Considering I had already done this multiple times, I guess a reboot sometimes resolves the issue? Fast startup is disabled both in Windows and on the server's motherboard, and for each reboot, I also tried to shut down and then cold start. Of course, as luck would have it, I have not seen a reboot fix the issue while I've been standing in front of it.

Repeat attempt #9 on a loop just to keep our operation running until I have more time to investigate the issue.

We did not have this issue with build #3030 or #3640. I tried to roll back to #3640, which seemed to be stable for us; but installer files I had previously downloaded and kept for older versions now all install the latest build, so as far as I can tell it seems impossible to roll back. I am aware of the known issue reported here, but that doesn't seem to apply to us.

• That bug is reported to have been fixed in build #3900, but the issue I'm having seems to have started with build #3900. Updating to #4100 has also not fixed the issue.
• That bug is reported only to apply to tasks with 500+ machines. We only have 300 machines total, and no more than 130 on a single task.

So far, whatever causes the issue to start has seemed to happen overnight such that it works one day, then we come in the next day and can't see our tasks. But recently, I happened to be in the process of doing deployments while everything was working, and then on the Deployment Tasks screen I saw an error message pop up with this information in the "More details" section:

HTTP transfer error

    Error code: 0x2470001
    requestType = "GET",
    uri = "Acronis6b/api/asd/v2/tasks",
    host = "Acronis6b",
    port = 8,081 (0x1F91),
    curlResult = 7 (0x7),
    $module = "patcher_remote_vsa64_3900"
    Http::Curl::CurlDetails::RequestScope::MakeError: <link url='execute://%VS_PATH%\Common7\IDE\devenv.exe @c:\jenkins_agent\workspace\asd-asd\3900\product\enterprise\common\http\curl_transport.cpp /command "Edit.GoTo 418"'> 
c:\jenkins_agent\workspace\asd-asd\3900\product\enterprise\common\http\curl_transport.cpp(418)</link>
Failed to connect to Acronis6b port 8081 after 1012 ms: Connection refused

    Error code: 0x2260007
    $module = "patcher_remote_vsa64_3900"
    Http::Curl::CurlDetails::RequestScope::MakeError: <link url='execute://%VS_PATH%\Common7\IDE\devenv.exe @c:\jenkins_agent\workspace\asd-asd\3900\product\enterprise\common\http\curl_transport.cpp /command "Edit.GoTo 417"'> 
c:\jenkins_agent\workspace\asd-asd\3900\product\enterprise\common\http\curl_transport.cpp(417)</link>

This error message only popped up once before my tasks disappeared, and has not popped up again. After seeing this error, I tried disabling the firewall on my machine and then rebooting; but that didn't change anything either.

You can see from that error message that I was running build #3900 at the time. I did upgrade to #4100 when it was released, but the issue persists. I poked around in logs in C:\ProgramData\Acronis\DeployServer and found this block repeated several times (presumably each time I attempt to view Deployment Tasks):

=================================== Mon Nov 21 10:16:01 2022
[266]<2429156451700> .   ; #179 [1508] Web Server: ListTaskHandler: Request Handling START ------
[266]<2429156467200> .   ; #180 [1508] Web Server: ListTaskHandler: Uri Path: /api/asd/v2/tasks
[266]<2429156472300> .   ; #181 [1508] Web Server: ListTaskHandler: Uri Query: 
[266]<2429156474300> .   ; #182 [1508] Web Server: ListTaskHandler: Method: GET
[266]<2429156476200> .   ; #183 [1508] Web Server: ListTaskHandler: Request Body: 
[453]<2429341866000> !!! ; #184 [1508] Web Server: ListTaskHandler: Handling request error: { Code: 0x640101, Line: c:\jenkins_agent\workspace\common\asd-asd\4100\p\products\patcher\libs\common\include\asd\client_server_connection.h:73, Text: "An internal error has occurred. Check the event log for more information." }.
[469]<2429355206100> .   ; #185 [1508] Web Server: ListTaskHandler: Request Handling START ------
[469]<2429355213400> .   ; #186 [1508] Web Server: ListTaskHandler: Uri Path: /api/asd/v2/tasks
[469]<2429355216300> .   ; #187 [1508] Web Server: ListTaskHandler: Uri Query: 
[469]<2429355218200> .   ; #188 [1508] Web Server: ListTaskHandler: Method: GET
[469]<2429355221300> .   ; #189 [1508] Web Server: ListTaskHandler: Request Body: 
[625]<2429515467300> !!! ; #190 [1508] Web Server: ListTaskHandler: Handling request error: { Code: 0x640101, Line: c:\jenkins_agent\workspace\common\asd-asd\4100\p\products\patcher\libs\common\include\asd\client_server_connection.h:73, Text: "An internal error has occurred. Check the event log for more information." }.
[641]<2429527100200> .   ; #191 [1508] Web Server: ListTaskHandler: Request Handling START ------
[641]<2429527109400> .   ; #192 [1508] Web Server: ListTaskHandler: Uri Path: /api/asd/v2/tasks
[641]<2429527112000> .   ; #193 [1508] Web Server: ListTaskHandler: Uri Query: 
[641]<2429527113800> .   ; #194 [1508] Web Server: ListTaskHandler: Method: GET
[641]<2429527117000> .   ; #195 [1508] Web Server: ListTaskHandler: Request Body: 
[781]<2429681938500> !!! ; #196 [1508] Web Server: ListTaskHandler: Handling request error: { Code: 0x640101, Line: c:\jenkins_agent\workspace\common\asd-asd\4100\p\products\patcher\libs\common\include\asd\client_server_connection.h:73, Text: "An internal error has occurred. Check the event log for more information." }.

I looked in Event Viewer, but didn't see anything happening at the times these errors were generated. I tried to find the path specified in the error to see if there were more logs I could look at, but C:\jenkins_agent doesn't exist on my drive, so I don't know where to go from here.

I am out of ideas. Any help is appreciated. I have not reimaged the machine this time; it is currently running build #4100 on the latest version of Windows 10 LTSC and not showing tasks, so I am able to run any suggested diagnostic tests.