I can load my library, but I cannot unload it with LibraryUnload. I do have a void WolframLibrary_uninitialize(WolframLibraryData libData) (which does nothing). I get the following error:

LibraryFunction::unloadlib: The library "/my/library.so" cannot be unloaded.


I don't seem to be able to unload not even the demo library:

LibraryLoad@"demo"


This gets me the same error.

EDIT: As worked out by Szabolc and halirutan, I need to load one function at least (LibraryFunctionLoad) to have the initialization function called, and so being able to call LibraryUnload successfully. Yet, my ultimate goal is to be able to reload a new build of the library: this seems impossible because calling LibraryUnload doesn't actually close the handle of the library, and further calls to LibraryFunctionLoad reuse it. I checked with lsof, and if I delete the library and try to reload it I get that the new file is ignored and instead a handle to the old file (tagged correctly as DEL in lsof) is kept.

-
Apparently I cannot unload the demo libraries too. I need this feature because I'm coding this library, and I don't want to restart the kernel each time. – Lorenzo Pistone Dec 5 '13 at 23:25
If you see the issue even with the demo-libraries, can you please make a minimal, self-contained example where you exactly show how you load and unload the library-function? I have Mathematica 9.0.1 and Ubuntu x86-64 and here, unloading seems to work properly. – halirutan Dec 6 '13 at 0:54
It seems that if I actually load some functions from the library (as here) then afterwards I can unload without problems. But if I don't load any functions (and only use LibraryLoad) then I get the error you mentioned. – Szabolcs Dec 8 '13 at 18:28
I'm not sure, but I think that LibraryLoad won't actually run the library initialization code, it will only load the library. You can use LibraryLoad to load any shared library that your code may need, not only LibraryLink-specific libraries (see here). So it would make sense that it doesn't attempt to run the wolfram library initialization. Maybe if this is not run then uninitialization fails. Just guesses. – Szabolcs Dec 8 '13 at 18:31
I'm not familiar with the intricacies of shared libraries, but when I was writing LibraryLink-based tools, as far as I can remember I didn't have a problem with using LibraryUnload, deleting the library, recompiling, and loading again. Are you saying this doesn't work as expected? – Szabolcs Dec 8 '13 at 18:53

Todd Gayley once mentioned that LibraryLoad is for pre-loading depend libraries you need. So for instance, if you write a function which needs some numeric-library, you can load this before calling the functions in your library.

Therefore, I believe the solution to your problem is that you have to call LibraryFunctionLoad on one of the functions you are developing and after that LibraryUnload should work as expected.
Furthermore, you should unload all functions with LibraryFunctionUnload before unloading the library.
I can confirm that this is how it works. Yet, LibraryUnload doesn't close the handle to the library, and if I replace it with a new build, there's no way to unlink the old version from the Kernel. – Lorenzo Pistone Dec 9 '13 at 13:37
Ok, if I LibraryFunctionLoad then LibraryFunctionUnload a function in my library, the library is correctly unlink from the kernel. If you add this to your answer I'll flag it as accepted – Lorenzo Pistone Dec 10 '13 at 11:46
@halirutan: you mention that LibraryLoad is needed to pre-load libraries which own libs depend on. This is what I currently need to do, but then I can't unload those libraries (there is no LibraryFunction to load from those). Do you have a solution how I can Unload those dependencies so I can remove them without closing the kernel? I'm afraid that won't work, which I consider an oversight (bug?)... – Albert Retey Dec 30 '15 at 16:58