Server performance guide

This is Ras' guide to performance for Pixelmon servers, now expanded and updated for general modded server admnistration. This page contains an accumulation of tips and common methods, or standards, to improve a Pixelmon server's performance. This guide therefore covers world pre-generation, launch Java arguments, recommended settings, commonly encountered issues, and lastly, discusses tools by which you can keep track of your server's performance, including reading and sharing logs. If you are installing Pixelmon on your own server, see the Server Installation guide instead.

This guide assumes that you understand the basic principles of Minecraft server hosting, and specifically, current Pixelmon server environments, including hybrid and Forge-dependent server APIs such as Arclight and Forge. If you do not understand these environments, or have never used them, we strongly recommend you read our Server Installation guide first, and if you need further help, please use #server-support in the Pixelmon Discord.

1.16.5 Server API Environment

Pixelmon servers are Forge servers, meaning they can use certain Forge mods server-side, allowing them use of features and utilities for server management. However, servers that require the usage of Spigot plugins may switch to using hybrid environments such as ArcLight, allowing them to run a patched Forge server which loads Spigot-based plugins.

World Pre-Generation

Please also note that this cannot be avoided. It is of extreme importance for servers to pre-generate their world's chunks as soon as possible. Failure to do so will greatly increase the odds of major, disruptive latency to gameplay.

1. In your server.properties (located in the server's root folder), set the setting max-tick-time to -1. This will prevent your server from timing out during this process. This setting is used to tweak the Server Watchdog, a Minecraft ‘feature’ that forcefully crashes your server if a single tick takes too long. Disabling it removes the risk of the Watchdog crashing your server while pre-generating.
2. If not already the case, whitelist your server and ensure all players have logged off. The process takes a few hours (depending on your server's hardware), and runs smoother if there are no players online.
3. Find, download and install a world pre-generation mod or plugin, such as Chunky (mod) or Fast Chunk Pregenerator (plugin).
4. Using Minecraft's vanilla worldborder commands, run /worldborder center [center-x] [center-z] (make sure to use your spawn coordinates) then use /worldborder set [diameter-in-blocks], replacing each variable with the desired configuration. The bigger the [diameter-in-blocks], the bigger the border, the bigger the map, the longer the pre-generating process will take. Once you have decided on your border's size, add 2000 blocks more to its count. If you want 15000, then 17000 will be your border size for pre-generating.
   1. The desired choice will vastly depend on your particular case. For starting, public servers, we recommend anything between 15000 and 20000.
   2. In general, anything above 30000 in diameter will be a difficult to manage, as worlds of that size can go upwards of 50GB in disk space after 10 months of use.
   3. This pre-generation process has to be repeated for every world that will be used by players. Vanilla Pixelmon servers have 4 by default - overworld, DIM-1, DIM1, drowned_world and ultraspace. Note, each world can have it's own specific size, but all will weigh heavily on disk space.
   4. When deciding your world's size, please take into account the need for fast, reliable and varied backups. Most servers will need at least one current, zipped backup of the world folder, as a safety net.
5. Assuming Chunky (mod) is installed, use /chunky worldborder [world] to get the world border you made, then use /chunky start. Use any equivalent command from your installed pre-generation plugin/mod to start pre-generation. You will see messages in console indicating the progress. Don't worry, it'll stop once it reaches your border, so you can basically leave your server empty and online until it is done.
6. Expect the pre-generating process to take a long time. Make yourself a coffee, or two, or three. Wait.
7. Once the completion message is shown in console, use /worldborder set [diameter-in-blocks] again. This time however, the [diameter-in-blocks] will be your initial size, without the additional buffer recommended above. This step will ensure that players at the world border won't be able to force-generate chunks past the border, as their view-distance has been accounted for in our initial pre-generating task.
8. Use Minecraft's worldborder damage command, /worldborder damage buffer <sizeInBlocks>, to turn on world border damage to values higher than default.
9. Repeat steps 4 through 7 for every existing world.
10. Once you are done, go back to your server.properties and set the max-tick-time to 18000. This re-enabled Minecraft's watchdog, ensuring that if your server stalls because of a memory leak, it will crash without preventing recovery.

Java Launch Arguments

1. In our Server Installation guide, we covered start.bat scripts. In that script, replace with the following:

java -Xms6G -Xmx6G -XX:+UseG1GC -XX:+UnlockExperimentalVMOptions -XX:MaxGCPauseMillis=100 -XX:+DisableExplicitGC -XX:TargetSurvivorRatio=90 -XX:G1NewSizePercent=50 -XX:G1MaxNewSizePercent=80 -XX:G1MixedGCLiveThresholdPercent=50 -XX:+AlwaysPreTouch -jar server.jar

1. Replace server.jar with your running jar. For example, if running a Forge server, your running jar is named similarly to forge-1.16.2-36.2.34-universal.jar.
2. Most Pixelmon servers will need at the bare minimum 6GB of RAM to run properly. Processes like pre-generation can exacerbate memory problems, which is why it should be done before your server's launch. The more players you have online, the more 'heavy' your server will be, so plan accordingly. For the average server with a small player-base, 8GB of RAM will suffice.
   1. More RAM does not mean more performance. In fact, on the contrary - it will worsen Java's garbage collection. Do not assign over 16GB, unless otherwise indicated by your particular circumstances and hosting experience.
   2. For a longer, and more technical read on Java launch arguments, we recommend studying Aikar's Garbage Collection guide, which applies to Forge, Sponge and Spigot environments alike.

Reduced View-Distance

1. Open your server.properties file, located at the base (root) folder of your server).
2. By default, Minecraft's view-distance is of 10. This means that around the player, a radius of 10 chunks will be loaded as they move/idle. That's a lot of chunks.
3. The more chunks are loaded, the worse it is for your server. Decrease the value of the view-distance to 6, or 4.

Tile-Entity and Entity Activation Ranges

Server APIs, like Sponge and Spigot (hybrids included), provide additional configuration files to fine-tune server variables. Usually named "global.conf" or "global.yml", a section of them should allow you to change activation ranges of both entities and tile-entities. This helps the server process less tasks per tick, and should greatly improve your performance. Note: an ‘activation range’ is the radius around the tile-entity (for example, a furnace) or an entity (for example, a creeper) for which it will tick when in proximity to the player. If you set your radius for example to 1, it will mean that your furnaces will only update when at 1 block around the player; of course, that also means your players will be very annoyed when their farms don’t update because the radius is too low. Similarly, if you have a creeper with an activation range of 0, chances are it will never update, and just explode when the player is standing right on top of it.

1. Open your server API's global config.
2. Search for terms such as "entity-activation" and "tileentity-activation".
3. Set the default range to 32 for all listed values.
4. Save and upload the file to your server, making sure to restart.
5. Periodically check that file over the next couple of hours or days (depending how active your players are), making changes where values could be raised higher, or lower, based on player-feedback and gameplay experience.

By lowering or raising the ranges of the blocks and entities defined in the global config to 16/8/6, you will be saving on processing power in exchange for 'timed out' functionality on the blocks and entities targeted. Note, the lower the value, the less strain those items have on your server, but that doesn't mean you should lower it to almost nothing. That would cause your entities/tile-entities to malfunction. We don't want that. Global config files are usually kept 'online' by the server, and should not be reloaded during run time, therefore, it is recommended to edit these files while offline.

Choosing your Plugins or Mods

This comes down to the quality of what your server is running. Not all plugins, or mods, have the same impact on your performance. Some, like Pixelmon, are unavoidable costs on our networks, but it might not be the case with plugins, or mods. In comparing Polis (Sponge plugin) to GriefDefender (Sponge plugin), the former is heavy on the server because it logs everything into a single file, whereas the latter divides the checks into many files that are managed by Luckperms. Choosing the right plugins (and mods) for your server will be one of the ways that you can greatly hinder, or improve, your performance. If you have questions about something, ask around, see if anyone has commentary on its performance on a server.

The golden rule is this: nothing is free. If you have a plugin with an inventory GUI, or block-logging features, or even regeneration, that stuff CAN be expensive. You have to be prepared for features added to impact your server’s performance. Remember, Minecraft is a single-core game, and no amount of additional RAM will make it go any faster on the main thread. Choosing your plugins and mods wisely will help you decide what you keep, and what is too costly to run.

Known Performance Plugin/Mod Configs

There's always new features for you to test and experiment with. The best way you can keep a good server performance is to watchout for new methods of lowering your server's strain. Of course, please ask around if you are going to edit something you have no idea about. Here are a few I've found.

1. Lighting async values can be found in Sponge's global.conf. You can assign more threads to this, and it should help with the lighting updates within loaded chunks. If you don’t know what this implies, or what it can potentially do, please do not use this option.
2. Real-time TPS is an experimental feature in Sponge's global.conf as well. In that file, under "modules", set real-time to true. It'll try to replicate as much as possible 20 TPS (ticks per second) for the client, even when the server is experiencing lag.
3. Sometimes, GriefPrevention (or GriefDefender) causes issues with the buildup of dropped items. Use /cf collide-block minecraft:item minecraft:any true override in your wilderness to lower their impact on the server.
4. Use CatClearLag, or a command scheduler, to queue /pokekill and dropped-item despawning. Both are notoriously fickle, so I recommend giving them the extra help needed to make sure those are recycled.
   1. CatClearLag, along with other lag-fixing plugins are known to work off of basic config fixes that you could be doing yourself, like the spawning rate in Pixelmon, or base MC mobs. I personally would not recommend using it as an ‘improving’ plugin, as it does not actually clear your lag. That said, if you need the entity clearing scheduling because, for instance, entity despawning is an issue, use it. It’s a good way to schedule clearing tasks, but don’t expect it to be magic. It won’t replace anything in this guide.
5. World saves are a real issue on any server. By default on Sponge, they happen every 45 seconds. In your global.conf, change the tick count between player-data and world save to the same amount - I usually do every 5 minutes, so 6000 ticks.
   1. You could be scheduling a longer interval, it’s a matter of personal decision. Just remember to think of this interval as “the maximum amount of timed progress my players are willing to lose in the instance of a crash”. If the server crashes, the data is reverted to the last world save - so you’d want these as frequently as can be managed.
6. In Sponge’s global.conf, enable structure saving and let it auto-populate to true. It’ll help saving mineshafts.
7. Sponge also has a use-real-time tick option, as well as use-panda-redstone and eigen-redstone fix option. All three are recommended. If you experience weird behaviour with redstone, please try first disabling the two fixes mentioned above to see if the issue persists. If it fixes the glitches you see, please report your findings to Sponge's tracker.

Pixelmon Performance Settings

1. Pixelmon Mod's Better Spawner is a treasure for performance. Play around with the different options, and see what works best for you. All servers are different, and all players want different things. It's up to you to find a good balance.

Keeping Track of Performance

Performance is never assured, and it will vary constantly, so you have to be prepared to recognize and ascertain issues as they arise. You can do this by monitoring your server, and keeping an eye on errors, or values out of the ordinary. Below is a list of the resources I personally recommend, and it includes how exactly I use them in my daily monitoring.

1. Sponge Timings: In your Sponge global.conf, under the "timings" section, set the history-length to 18000, and the interval to 2000.
   1. This will allow more data to collect in your Sponge Timings, which you can view by using /sponge timings paste. Though this might look complicated at first, Sponge Timings are a useful tool to find out performance issues. Make sure that the right-corner header is set to "AVG" and "ALL". Intuitively, the graph should be mostly green, with no major fluctuations. Pictured on the right-hand side is an example of this.
      

      Pictured: Sponge's Aikar Timings catching classic world-save pokemon performance issues.
   2. The data under it, sorted into a table, should not contain values out of the ordinary. If you see "doChunkMap", "chunkGenerator" or "chunkPopulate", you didn't fully pre-generate your map.
2. Nucleus’ GC Command: Nucleus’ /gc -c command is a great way to quickly assess performance in the last session. Always note the average "loaded chunk" count, which will vary depending on the number of players.
3. Forge & Sponge’s TPS Command: Both Forge and Sponge have a command to monitor TPS per world. Use /forge tps or /sponge tps to quickly check the server's current performance. Minecraft counts 20 ticks per second (TPS), so if the resulting output shows it to be below 20 TPS, run a timings and check your console for clues as to what is happening.
4. Server Console: the most primitive tool left, the server console, is only ever useful when you see a recurring error appear as your TPS decreases. Most, but not all, spamming errors will affect your performance, so make sure that you have checked for errors in the console before further investigating.
   1. If you see "moved too quickly!" messages, then your server is suffering from chunk loading. Make sure your map is pre-generated to lessen the impact of chunk loading.
5. TickProfiler: pretty simple tool to use, it is installed as simple jar. You can find the wiki and it’s open source on MinimallyCorrect’s github. It will be a useful tool that can help when timings do not seem to give you a good idea of what exactly is causing all that TPS loss. It is used to track entity and tile-entity usage. Thanks to merjilin for this tip.
6. WarmRoast: Recommended only as a profiling tool NOT to leave online at all times, you can download and install it from its author’s Github, sk89q. This is mostly useful when you have a memory leak and timings cannot seem to decide what’s wrong.
7. Spark: this plugin for Sponge was developed by Luck, the same person that made Luckperms. A more user-friendly option to WarmRoast, this allows process profiling in-game (which WarmRoast does not). The added accessibility makes this my preferred option, as it is easy to setup and has a discord if ever you need more help. Download and read the wiki here. A simple command to use is /spark sampler --thread * on. This sampler will have to be stopped once you’ve neared 3-5 minutes of runtime during lag spikes.
8. Sampler: arguably the most useful out of the bunch in my personal experience, Sampler allows server owners to set a TPS timer and catch an all-threads, spark-like memory snapshot that is registered to a file. It also automatically will generate stall-reports, useful for never-ending spikes, and effectively replacing that ‘watchdog’ crash report that is pure garbage (sorry to burst anyone’s bubble). It’s a free, public plugin. Simply drop it to your /mods folder to install.
   1. To get a sampling report, run /sampler trigger ticktime 30000. This command will therefore take a snapshot nps when the ticktime reaches 30000. If you want to be more aggressive, you can always set this lower. You will have to do this every restart (and note, as all sampling plugins, running it does impact your performance slightly, so I wouldn’t run it just for fun). That command will generate a .nps file in your server’s root directory.
   2. To open it, you will have to download VisualVM (free software). Once you do, you can unzip to your desktop the downloaded file, and run the .exe.
   3. Once you have VisualVM open, you can open your .nps file, and start poking at the data collected.
   4. My personal trick is to go into ‘Select Threads’, uncheck everything in the dropdown list and keep ‘Server_Thread’ checked. The data will then be re-sorted automatically.
   5. Just like a Spark, you can then collapse and expand tabs to look at the possible lag culprits, like this.
   6. As mentioned previously (and if you didn’t see this, read back up), stall-reports are created automatically. They are located in the ./stall-reports folder in your server’s directory, and they are simple text files, like crash-logs or debug logs.
   7. Download and start opening them (personally recommend Notepad++), and check for the first sections of the stall report to see the possible sources.
   8. Though this is slightly less accurate than sampling reports, it can be useful in determining a slowdown in console, and usually will be created in batches. For better accuracy, consider using the first stall-report out of a single ‘batch’, since all the subsequent stall-reports are usually slowdowns caused by the initial performance issue.

Crashes, Timeouts, TPS Loss and Other Painful Matters

There is no such thing as perfect stability. Issues always arise, and when you start noticing that your performance is going downhill, act fast. Certain issues, like the ones caused after crashes, can cripple your server and your players, making it hard to recover. In the interest of not cluttering endlessly this section, only common Minecraft/Forge/Sponge issues that are not plugin related will be covered. Here are a few of the common crash issues, and how to solve them.

1. Corrupted regions or playerfiles
   1. A region file may be completely broken, to the point that it is best to delete it. In your ./worldname/region or ./worldname/playerdata folder, please check for any region files (.mca files) with a size of 0. If there are any, delete or restore from backup. These are broken files, and they will crash you if the world save tries to process them.
   2. Sometimes, chunks may not appear problematic at the first glance. Use Fenixin’s Region Fixer to run your world through the checkup process. Any issues it might find with the files will point you to regions that are broken.
2. Corrupted or malformed level.dats
   1. Probably the scariest issue out there for most, this one has the particular ‘feature’ of completely changing your modded block IDs to other blocks. Like all Apricorn trees might become Bone blocks, and so on. To recover:
      1. Shutdown your server.
      2. From a backup, grab the level.dat, level_sponge.dat, level.dat_old, level_sponge.dat_old and the forcedchunks.dat. Of course, if you don’t run a Sponge server, no _sponge.dat will be found.
      3. With the server still offline, replace all 5 files from your backup to your main server. If you have multiple worlds affected, repeat the process as needed, as these 5 files are different for each world.
      4. Start up the server. The blocks should be back to normal.
3. Server timeouts in 1.12+: Random Patches. If your server has player timeout issues, try this server-side mod, by TheRandomLabs. It should fix the Mojang hardcoded server timeout on login.