Page 1 of 2
FPS freezes for a while every some time
Posted: 12 Jul 2022 09:46
by azaz44
I run Heli-X on Windows, gaming laptop, VR, HP Reverb G2.
It is running smooth, but every couple of seconds it freezees for maybe a quarter of a second, and then it is smooth again.
So in general it works, but it is little annoying.
Anobody had similar issue maybe? I think my other software is fine, because I don't have it in other sims I use, but I'm open to any suggestions on tracing the problem.
I tried to change java process priority to high, but this didn't help.
Re: FPS freezes for a while every some time
Posted: 12 Jul 2022 17:19
by azaz44
I made a bit of investigation. I ran Process Monitor because I thought that these "glitches" happen because maybe Heli-X loads some files, like sounds or whatever. Especially because they happen more often right after you start it, and it gets more stable after a couple of minutes.
I didn't find any of that, but anytime a "glitch" (it freezes for a couple of frames) happened, there was "Thread Create" operation logged in Process Monitor.
Not every "Thread Create" resulted in a freeze. But every freeze resulted in "Thread Create" being logged.
is it maybe Java garbage collection triggering?
- Screenshot 2022-07-12 181840.png (63.08 KiB) Viewed 19103 times
Re: FPS freezes for a while every some time
Posted: 13 Jul 2022 12:19
by Michael
Hi,
what you can try:
Try a small resolution and see if it works better.
Try without a model adn see if it works better.
in the startscripts, you could also play with java parameters:
-Xms512m gves ths initial stack size, which could be enlarged.
I am not sure if it is the garbage collection. In the start script you could try different options:
E.g. -XX:MaxGCPauseMillis=10
https://docs.oracle.com/en/java/javase/ ... 670B44E49C
Does it help?
Re: FPS freezes for a while every some time
Posted: 30 Jul 2022 14:44
by azaz44
Hi Michael. Excuse me late answer, I thought I'm subscribed to this thread and will get email notification, but I probably wasn't.
I was able to trace gc with -Xlog:gc option, and it looks like it's not GC. There's nothing about GC when "freezes" happen. And also sometimes GC happens, but then nothing freezes.
I also tried -Xlog:all, but this is logging stuff all the time and it's hard to browse.
But increasing initial heap to 2048 or 4096 might help, I increase it and will keep an eye on it.
Re: FPS freezes for a while every some time
Posted: 31 Jul 2022 00:16
by azaz44
I made quite some research on this topic today.
It looks like the reason is JIT compiler. I added logging around threads (-Xlog:thread*=debug) and any time I get this "freeze" and FPS drop, I get something like this in the console:
Code: Select all
[123.000s][debug][thread,smr ] tid=35520: ThreadsSMRSupport::free_list: threads=0x0000018d0c5fd700 is freed.
[123.002s][debug][os,thread,timer] name='C2 CompilerThread2', exit-phase1=0, exit-phase2=0, exit-phase3=1, exit-phase4=2
[123.003s][debug][thread,smr ] tid=35520: ThreadsSMRSupport::smr_delete: thread=0x0000018d0311f800 is deleted.
[123.004s][info ][os,thread ] Thread finished (tid: 35520).
[123.120s][debug][os,thread ] Thread 71272 stack guard pages removed: 0x000000038f000000-0x000000038f004000.
[123.121s][info ][os,thread ] JavaThread exiting (tid: 71272).
[123.121s][debug][thread,smr ] tid=71272: Threads::remove: new ThreadsList=0x0000018d1637dd00
[123.122s][debug][thread,smr ] tid=71272: ThreadsSMRSupport::free_list: threads=0x0000018d0c7224c0 is freed.
[123.124s][debug][os,thread,timer] name='C2 CompilerThread1', exit-phase1=0, exit-phase2=0, exit-phase3=1, exit-phase4=2
[123.125s][debug][thread,smr ] tid=71272: ThreadsSMRSupport::smr_delete: thread=0x0000018d140ad800 is deleted.
[123.127s][info ][os,thread ] Thread finished (tid: 71272).
or this
Code: Select all
[111.440s][info ][os,thread ] Thread started (tid: 75588, attributes: stacksize: default, flags: CREATE_SUSPENDED STACK_SIZE_PARAM_IS)
[111.441s][debug][thread,smr ] tid=68832: Threads::add: new ThreadsList=0x0000018d04e60180
[111.445s][debug][thread,smr ] tid=68832: ThreadsSMRSupport::free_list: threads=0x0000018d0aa3c730 is freed.
[111.446s][info ][os,thread ] Thread is alive (tid: 75588).
[111.447s][debug][os,thread ] Thread 75588 stack dimensions: 0x000000166f200000-0x000000166f300000 (1024k).
[111.447s][debug][os,thread ] Thread 75588 stack guard pages activated: 0x000000166f200000-0x000000166f204000.
[112.795s][debug][os,thread ] Thread 75588 stack guard pages removed: 0x000000166f200000-0x000000166f204000.
[112.795s][info ][os,thread ] JavaThread exiting (tid: 75588).
[112.796s][debug][thread,smr ] tid=75588: Threads::remove: new ThreadsList=0x0000018d00d03790
[112.801s][debug][thread,smr ] tid=75588: ThreadsSMRSupport::free_list: threads=0x0000018d04e60180 is freed.
[112.802s][debug][os,thread,timer] name='C2 CompilerThread1', exit-phase1=0, exit-phase2=0, exit-phase3=1, exit-phase4=5
[112.803s][debug][thread,smr ] tid=75588: ThreadsSMRSupport::smr_delete: thread=0x0000018d0783c000 is deleted.
[112.804s][info ][os,thread ] Thread finished (tid: 75588).
and actually I was able to make Heli-X run smooth by adding this to java command line:
both affect how JIT works. It basically makes Heli-X stable (not doing a lot of JIT) very quickly.
Can also debug JIT with this switch:
Maybe you want to look at it. I think "-client" might be a better choice for Heli-X. The other switch might need some tweaking, 500 works for me although it is a bit aggressive.
Re: FPS freezes for a while every some time
Posted: 02 Aug 2022 08:07
by azaz44
More info
It seems adjusting JIT helped a lot, but it still has glitches sometimes. So I was maybe happy too early. What I found
- first of all, problem also happens in non-VR. I didn't notice before, because I fly VR 100% of the time.
- after adjusting JIT (like in previous post), and using `-Xlog:thread*=debug` it now doesn't show anything about threads in the console, when problem happens
- however Process Monitor shows clearly, that there is thread activity in such moments
- that's all I know as of now..
I was able to record it, see YT link:
https://youtu.be/24Q-wJp1xGE
at 0:24.
It looks like there is a 2nd problem, or there is one problem and server version of JIT was just triggering it more often.
Like if there would be some issue in Java anytime it creates a new thread. And with JIT adjustments to client version it just doesn't do this often (C2 server-version JIT was creating a lot of threads).
But of course I have no idea what are these threads which are causing issues..
Re: FPS freezes for a while every some time
Posted: 02 Aug 2022 10:22
by azaz44
Sorry for spamming, but I have more insight.
First of all, this is how it looks over time, just to have idea how often things happen. Every "Thread Create" was connected to FPS drop.
- Screenshot0.png (120.47 KiB) Viewed 18770 times
I was able to record thread activity and it seems anytime this happens, there's thread creation / destroying around these two:
- Screenshot.png (120.47 KiB) Viewed 18770 times
If I suspend these threads, it seems problem is gone. I will test over time, but I was running Heli-X for 15 minutes with no FPS drop and no new "Thread Created" events in process monitor. And before it never happened that it would run 5 minutes without a problem.
This is a workaround of course and maybe introduces some problems, not sure yet. I don't have time to research more at the moment, will research later..
Re: FPS freezes for a while every some time
Posted: 06 Aug 2022 07:27
by azaz44
Not a lot of new knowledge, but I can say now with confidence that suspending "TpReleaseCleanupGroupMembers" threads on Heli-X makes Heli-X run smooth.
Also, what this function is doing is release a thread pool (Tp = thread pool).
I haven't found a way to automate this yet, so I do this manually anytime I start Heli-X.
Why would the process release a thread pool every couple of minutes - this looks wrong. Not sure if it's a problem in Java, or some library, no idea.
Re: FPS freezes for a while every some time
Posted: 15 Sep 2022 09:23
by azaz44
Just as confirmation, this really works. I always suspend these "TpReleaseCleanupGroupMembers" threads after starting Heli-X and this makes it run smooth. It would be great if this problem could be somehow addressed, because it's a super long procedure to get Heli-X running this way...
Re: FPS freezes for a while every some time
Posted: 07 Feb 2023 19:04
by azaz44
It's great to see 10.1 version!
But this problem is still there. I workaround it by suspending threads every time I start Heli-X, but this is very annoying.
I wonder if I can help somehow to track this? Or is this problematic to reproduce?