tag:blogger.com,1999:blog-6459030366729235821.post7951593557438887622..comments2014-04-17T15:29:39.159+01:00Comments on Are We There Yet: A Garbage Collection Analysis of PCGen, the popular Open Source Character GeneratorMartijn Verburghttp://www.blogger.com/profile/01458162075331573781noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-6459030366729235821.post-31395321708096367512014-04-17T15:29:39.159+01:002014-04-17T15:29:39.159+01:00Unfortunately not no, we'll likely plan for th...Unfortunately not no, we'll likely plan for this next year.Martijn Verburghttps://www.blogger.com/profile/01458162075331573781noreply@blogger.comtag:blogger.com,1999:blog-6459030366729235821.post-18594139700202817072014-04-17T14:41:08.639+01:002014-04-17T14:41:08.639+01:00Does your censum tool analyze logs from IBM JVM?Does your censum tool analyze logs from IBM JVM?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6459030366729235821.post-89995116252919157622013-01-21T17:19:54.367+00:002013-01-21T17:19:54.367+00:00Hi Will,
Thanks for your comment. So the code ver...Hi Will,<br /><br />Thanks for your comment. So the code very roughly:<br /><br />1 Has a handle on the text file (a .lst proprietary format)<br />2 Loads all of the data into memory<br />3 Updates a Character Document Object Model (CDOM) with the data<br />4 Keeps some data cached as pure reference data (like a database table with static data, but in memory)<br />5 Releases the handle to the text file<br /><br />At this stage all of the data is needed to be kept in memory in order for the CDOM to correctly model the character as it gains and loses levels, as well as having classes, abilities, templates, feats, spells, equipment and much more change its stats!<br /><br />So it's hard to see what we could remove in terms of holding references - but we certainly can reduce the size of the overal CDOM per character as well as the static data...<br /><br />It's a fun engineering effort for the team!Martijn Verburghttps://www.blogger.com/profile/01458162075331573781noreply@blogger.comtag:blogger.com,1999:blog-6459030366729235821.post-20275012347844870982013-01-18T23:33:21.019+00:002013-01-18T23:33:21.019+00:00It looks like you are loading a lot of those text-...It looks like you are loading a lot of those text-based config files and holding on to them. If you were more aggressive about removing references to the text config files in memory, that should help...?Willhttps://www.blogger.com/profile/10245838878774225257noreply@blogger.comtag:blogger.com,1999:blog-6459030366729235821.post-49164059456354089952013-01-18T10:54:01.408+00:002013-01-18T10:54:01.408+00:00Hi Ryan,
Thanks for commenting!
Log vs Realtime ...Hi Ryan,<br /><br />Thanks for commenting!<br /><br />Log vs Realtime processing:<br />------------------------<br /><br />Just to be clear - you can feed the log into a number of tools or even just read it yourself if you have the time and the experience. It certainly doesn't have to be fed into Censum!<br /><br />You're quite correct that post processing a log is not the only way to determine how to tune a GC. But trying to tune 'live' or in real time GC information has several disadvantages:<br /><br />* Connecting to a live running JVM using the GC MXBeans actually creates its own objects (and therefore garbage) impacting the very application you're trying to tune.<br /><br />* The MXBeans don't give you as much information as the logs can (it's a limited API).<br /><br />* Real time analysis only gives you a short window for analysis, so you don't see the full picture of how the memory in your application is behaving over it's natural cycle (e.g. A payroll application over the period of a day's work, or even a week given the Friday afternoon timesheet rush). Tuning based only on partial data can cause major problems for your application.<br /><br />Other tools vs Censum:<br />--------------------<br /><br />GCViewer is pretty decent, as is HPJmeter - but when we polled large numbers of developers and operations folk, we discovered that a majority of them didn't have the time to learn how to interpret the results that those tools give out. They were also often dealing with an emergency production issue and wanted the tool to tell them exactly where the problem was and what to do about it.<br /><br />We're trying very hard with Censum to give that definitive answer (as much as is mathematically possible) and make a solid recommendation (e.g. Set your NewSize=X). We still have some ways to go before we can recommend in all cases, but Censum is constantly learning and we're confident the accuracy and range of analytics will continue to improve as the machine learning and modelling we use gets more data.<br /><br />G1:<br />---<br /><br />Your comments here are really interesting - we haven't heard those levels of confidence about G1 from anyone, even including the Oracle guys themselves who wrote it! G1 has only just gain official blessing from Oracle as a supported GC - we'll be watching it with interest but we have our doubts that it will 'automatically' fix everything. Not even Azul claims that their pauseless C4 GC does that :-).<br /><br />I noticed the GCViewer actually has some G1 support in it - so I guess it's still of need/interest to analyse G1 and tune it.<br /><br />OpenJDK and the Java community at large could really benefit from your findings, please do post them to the GC and hotspot mailing lists! If G1 can be proven to be in almost all cases better than existing collectors, that's a great leap forward for everyone.<br /><br />Thanks again and if you'd like an eval copy of Censum, let me know :-).Martijn Verburghttps://www.blogger.com/profile/01458162075331573781noreply@blogger.comtag:blogger.com,1999:blog-6459030366729235821.post-65122312237211014752013-01-18T04:35:52.625+00:002013-01-18T04:35:52.625+00:00I understand this is a blog post meant to shill Ce...I understand this is a blog post meant to shill Censum - but your point that being able to tune GC using only GC log files is totally wrong... especially considering the fact that you feed GC log files INTO censum. <br /><br />The free open source GCViewer doesn't have the same "wizards" that Censum has, but it does provide a very good summary of the pause times from a GC log and it makes it very easy to tune an app's GC performance. <br /><br />With the results I've been seeing as we've been moving towards using G1 in production with our apps, G1 may make a lot of GC tuning a thing of the past (so by the time Censum would really hit traction in the market, G1 might eliminate the need for such a tool)Ryanhttps://www.blogger.com/profile/02926383743502657473noreply@blogger.com