Recommendations for managing multiple plugins & third party dependencies
- 1 Setup a Project multiple people can work on
- 2 Use third party dlls
- 3 Use shared Projects
- 4 Optimize your Build
Setup a Project multiple people can work on
The main problem you will encounter if you have several people working on the same source is the differences between working environments of the different people.
- Location the Project is checked out to
- Location of the Torch installation
- Location of the Dedicated Server files
- Location of dependant projects
play a big role in working together. But that must not necessarily be the case when working with multiple people, you are also dependent on your own machine not changing at all, unless you want deal with fixing your References again after a windows reinstall.
Introducing THE Batchfile
To ensure everyone works under the right conditions you should check in a simple batch file into projects Github or other version control system. Everyone that wants to work with your project has to execute this file once.
Make sure to name it pretty obvious like Setup (run before opening solution).bat
1 :: This script creates a symlink to the game binaries to account for different installation directories on different systems. 2 3 @echo off 4 set /p path="Please enter the folder location of your SpaceEngineersDedicated.exe: " 5 6 cd %~dp0 7 rmdir GameBinaries > nul 2>&1 8 mklink /J GameBinaries "%path%" 9 if errorlevel 1 goto Error 10 echo Done! 11 12 set /p path="Please enter the folder location of your Torch.Server.exe: " 13 cd %~dp0 14 rmdir TorchBinaries > nul 2>&1 15 mklink /J TorchBinaries "%path%" 16 if errorlevel 1 goto Error 17 echo Done! 18 19 echo You can now open the plugin without issue. 20 goto EndFinal 21 22 :Error 23 echo An error occured creating the symlink. 24 goto EndFinal 25 26 :EndFinal 27 pause
This Batchfile creates symlinks to the folders your Torch.Server.exe and your SpaceEngineersDedicated.exe are located and places them into your project directory.
That way we avoid system differences. Because everything your project needs to do is to load references from its own local project directory. no need for Absolute paths or weird looking relative pasts as ../../../Games/Torch/Content.Sandbox-commons.dll
Be careful: this link will be a so called hard link. So it wont have the typical .lnk extension as normal links in you create on your desktop. All changes including deletion of the folder will affect the linked directory also. Use commandline rmdir to just remove the link without deleting everything.
Setup your projects according to the new paths
if you peek into your projects .csproj file using Notepad++ for example you most likely have something like this:
1 <Reference Include="NLog"> 2 <HintPath>..\..\Server\torch-server\NLog.dll</HintPath> 3 </Reference>
As said before this is a huge problem if the Server is located elsewhere for other people. Now that you have run the batchfile you can simply change it to
1 <Reference Include="NLog"> 2 <HintPath>TorchBinaries\NLog.dll</HintPath> 3 </Reference>
If everyone keeps to this format you can work with the same project file, without constantly having to deal with changes in your Project that look like:
Modify the Batchfile for your needs
Most of the time just having the game and torch directory linked is enough in order for you to work with your collegues on the plugin.
However depending on the growth of your plugin you may need to add additional links. For example do you have a shared project you want to include in your plugin? Better not try to directly access it because who knows where your collegues have it checked out to? Just add a new link and add it through that.
Alternatively of course you could use some build-systems for that. In Java you can use Gradle which via plugins also works for c# to set projects up. But depending on your project it may just be overhead you don't want to deal with.
Use third party dlls
Any third party dll you use (for example DSharpPlus.dll) in case you want to build a plugin that interfaces with Discord must be a Private reference (meaning the dll will appear in your build folder) and must be put into your plugins zip in order to be loaded by Torch for execution.
However be vary careful with what dlls you use. If an other Plugin uses a dll with the same name both will be loaded by Torch. However when loading the dll it will no longer contain the version information. So if you use Version 1.4.2 of that dll and the other Plugin uses 1.3.0 (so an older one) either your plugin or the other developers plugin may break since only the first loaded dll is used.
Ways around that
Rename the dll
The easiest way would be to just rename the dll you are referencing to. So you run a lower risk of conflicting. However: Depending on the case just renaming the .dll file itself wont do you would need to change the assembly name inside the dll which could mean you have to grab the source and compile it yourself.
Updating all Plugins when the dll changes
If it is for example your dll and you chance something to it you could just update all your Plugins to the new version to make sure everything is in order. However this could mean you have to update and test many plugins. And you probably don't have the time to test just everything.
If you have multiple plugins using the same dll you can also add a dependency plugin. So in order for your plugins to work you can add an other plugin as dependency that will be loaded first.
Torch wont automatically download missing dependencies but warn the user if its missing. Such Plugin can be used to get the necessary dll for you. But it still means that you potentially break your non updated working plugins if you do it that way.
You can add a Dependency by adding the following code to your Manifest
1 <Dependencies> 2 <PluginDependency> 3 <Plugin>23cb520c-ee5e-437b-bb6f-76e832c5027a</Plugin> 4 <MinVersion>126.96.36.199</MinVersion> 5 </PluginDependency> 6 <PluginDependency> 7 <Plugin>069b65fb-b47a-417d-b847-8f641974689a</Plugin> 8 <MinVersion>188.8.131.52</MinVersion> 9 </PluginDependency> 10 </Dependencies>
Of course in case you only need one Dependency you can remove the other. The important part is that the minimal version and the GUID are correct.
If the dll you want to include is your own and you want to use it on different plugins you will not spot a problem at first. However you will start getting problems once one of your plugins uses a later version of the plugin. And in worst case the server administrators notice it before you do.
If its your own code you can work around that using Shared projects (see below).
Surely after some time you will have developed a handy little toolkit with core features you want all of your plugins to use. These core features can be provided via dll but as mentioned before you will run into problems when your plugins use different versions.
Advantages over dlls
A neat workaround for that is the use of shared projects. Unlike regular projects shared projects don't build their own dlls and don't have any references to other projects or dlls.
Your shared project compiles when the project you want to use it with compiles. Has the same references and its code will be in the same dll.
So its perfect if you don't want to deal with multiple copies of the same classes in different projects. And since its all contained in one dll it won't be a problem if you change your shared project and only rebuild that one project that needs to be updated.
As you may have guessed this only really works if you have control over the source you want to share. You cannot just take any third party library and put it in as a shared project.
So they wont solve possible conflicts with other plugins for commonly used dlls, but conflicts with your own plugins which are far more likely.
How to set it up?
Shared projects seem to be a feature in Visual Studio that seems fairly new. Here is a tutorial on how to set it up and why to use them:
It was also tested with the latest 2019 community edition of Visual Studio and works just fine.
Optimize your Build
Just adding the references of Torch and Keen to your project will of course work, but after you built your project and plan to add them to your plugins zip file you will notice problems.
Nothing that can stop you of course but something that will eventually reduce your efficiency.
You are basically only interested in those two files but your build folder is littered with references and the references of the references.
Set your references to public
By default all references you add via Visual Studio are marked as private. Which in other words means those references are needed in order for your plugin to run and therefore have to be in the build folder.
Though it is correct if you were to add a main method to your project and run it stand alone you would need all that, this is not the purpose of your plugin. Your plugin is loaded by torch and torch needs all these dependencies to execute your plugin. So your build can fully ignore them.
You can set them to "private = false" either in the UI or your csproj file.
1 <Reference Include="NLog, Version=184.108.40.206, Culture=neutral, PublicKeyToken=5120e14c03d0593c, processorArchitecture=MSIL"> 2 <SpecificVersion>False</SpecificVersion> 3 <HintPath>TorchBinaries\NLog.dll</HintPath> 4 <Private>False</Private> 5 </Reference>
Which causes the compiler to only reference them in your build but not include them in your build folder. which then leads to a clean build folder where all files in there can go straight into the zip for release or testing.
Use Build-Scripts for your manifest.xml
If you have added your manifest.xml to your projects main folder you can further improve your build by just copying it to the output folder as well.
for that you can just create a post build script containing
1 xcopy /y /d "$(ProjectDir)manifest.xml" "$(ProjectDir)$(OutDir)"
of course using 7zip you can directly create the zip file for testing if you need to. But if creating and copying a zip file directly or not is completely up to you.
Only think you need to be careful of is to not use paths which are exclusive on your system if working with other developers. But if you have set up the hard links as shown above you have a direct access to the torch binaries folder which contains the Plugins you can directly edit if you like.
Automatically launch torch
Torch has a feature that allows you to set the custom location to load exactly 1 plugin. (the one you just build for example)
- Makes sure torch skips updates for itself and the game. Useful to prevent any unwanted updates that potentially break everything and stop you from continuing your work.
- Provides the path on where your plugin is. It does not need to be zipped for that.
Once gain make sure to use paths to torch every developer can use if needed. You can also start it via post build script if needed.
However it will only show the build as successful after Torch has been closed again. And you cannot build an other one while its open.
Since your manifest, dll and pdb would be in your build folder its the perfect target.
However if you want to test your plugin working together with other installed plugins you have to use the zip and copy to Plugins folder approach.
A Build-Script could include:
1 start "" "$(ProjectDir)\TorchBinaries\Torch.Server.exe" -noupdate -testplugin "$(TargetDir)
The missing " in the end is no mistake if you put it there torch wont be able to run as it magically creates its own closing ". Reasons for that are unclear.