Suitability of Mathematica as platform for engineering calculations and programs

Question

I am considering using Mathematica with Wolfram Workbench as a standard platform for calculations and programs in a large engineering department. I am looking for a solution that would provide better validation, documentation and revision control than Excel spreadsheets with VisualBasic macros. Additional benefits would be to use built-in functionality to reduce development effort and to allow engineers to use code interactively instead of in a black-box mode.

I have used Mathematica and WebMathematica for individual projects in the past, and am familiar with the various customer examples that Wolfram advertises. However, I don't have experience or examples of using Mathematica across a large engineering department.

I expect that we would develop packages for various back end functionality (likely on the order of 20,000+ lines of code), and users would access it either directly in the notebook, through CDF applications or through WebMathematica depending on the application.

Is Mathematica suitable for this type of use? Can average users learn Mathematica without too much trouble and become productive quickly? Is there a better solution you would suggest evaluating?

Answer 1

I'm late and most of what I'll write has in one or another form already been said in some of the comments. Nontheless I can't resist to provide an answer. I might add that I'm working for a german based engineering service provider (SmartCAE) and we are using Mathematica as one of our main tools for almost 15 years, so we do have some experience in the field.

Here are the answers to your specific questions from my point of view:

• From a pure technical point of view, Mathematica would definitely be suitable, and much more so than Excel.
• Learning Mathematica can very well be seen as a lifetime's task. On the other hand I think it's possible to get ready to solve typical engineering tasks within a day, depending a lot on prior knowledge about mathematics and other tools or programming languages. But Mathematica is quite different than many other systems/languages, so at some point many people start to struggle. It needs a "positive attitude" to get past that point, and only after that one will reach the full productivity that it can provide. I would expect only a fraction of the engineers in a large departement to reach that point, everyone else will learn enough to get their jobs done but probably not enough to recognize/appreciate the added power that Mathematica could provide.
• Considering engineering applications, I think there are alternatives, but I think they either do suffer from roughly the same weaknesses as Mathematica or are very (too) limited to what they were made for, which might or might not be a problem for your departement. Some of them might be more well known among engineers and thus have a lower acceptance threshold. From the pure technical point of view I'd consider Mathematica to be a good choice when compared to the alternatives.

For engineering applications both Mathematica and Excel do only provide a relatively low level platform, so you will probably need some additional tools to make your engineers productive. This could be a (or some) DSL as Leonid suggests, but maybe also just some other tools which will help the engineers to do their job without necessarily become software engineers and Mathematica experts. We have developed such tools for our internal usage, and without them would have a hard time to provide our services within the given cost limits. Actually I'd consider it necessary to provide such additional tools for Excel as well, but there some knowledge is implicitly expected from everyone working with a computer and people will happily (?) use and accept it as it is. That might be unfair but is a fact that simply can't be overseen.

That said, I think whether Mathematica can successfully be introduced in a larger departement which uses Excel at the moment is not so much a question of technical suitability. I'd rather see it as a "social task": Such a transition will force everyone to learn new things and will make experienced personell feel like beginners. Only if you can convince the key users ("tool providers") and a majority of the engineers that such a transition makes sense you can expect them to take that hurdle. If only some users with influence will not be convinced, I would expect such a transition to fail. As every software tool Mathematica has weak spots and oddities: these are usually easy to overcome and none of them does actually make Mathematica useless or very hard to use, but it's easy enough to make it look like that would be the case. Unfortunately it doesn't help that you can find such cases for the exisiting tools like Excel just as well, you can't expect people to make fair comparisons in such situations. So I think you'd really need to make sure that you have the support of users and the management. To gain that, you'll probably need to:

• Ensure that everyone agrees that there is a problem with using Excel, maybe listing everything that can't be done with it or has gone wrong in the past.
• Persuade that Mathematica will solve those problems without introducing new ones.
• Comparison with alternatives
• Have ready a plan how the transition could be done
• Have ready an estimation on what this will cost and what it might save/add in the future.

Especially the latter will need to be realistic, you'll most probably be cited on that. Costs will of course include training, licenses, expenses for rewriting existing tools, additional IT administration, etc.

A set of real world examples will help a lot, unfortunately many of the examples you'll find on WRI's pages are rather showing the principles, but don't really qualify as "real world" (IMHO). In many cases that's just because they avoid the "simple" things like importing and exporting data, providing numbers with units etc., in other cases they just don't provide all the parameters as inputs that you'd need to run real world examples. So you'd probably need to provide some examples yourself.

All in all that might be a big project. On the other hand, and that has also been mentioned in the comments, you could just as well try to introduce Mathematica as a tool in a less official way by just starting to provide useful stuff written with it. If you manage these tools to run in the CDF-Player, you could even give them away to your colleagues without causing additional costs. I think many "modern" tools have been introduced in such an "informal" way in large companies (I know some examples from german automotive companies, but unfortunately that hasn't happened to Mathematica in a larger scale (yet?)). Of course that's only possible if your employer allows it.

Answer 2

First comes a disclaimer that I don't have experience in introducing Mathematica to a large department, so you can take all I say with a grain of salt. That out of the way, I think, the answer depends on the level of preparation and willingness to learn new things of the average user in your department, but, given that you anyway plan to deploy a rather large code base, I would recommend to create a framework or DSL on top of Mathematica, which would isolate the average user from the full complexity of Mathematica.

If you make a DSL, you can as well code it in Mathematica, and make it strongly typed. This assumes a separation between users and developers of such custom language. Once the users request some new feature, it is implemented in the language and made available to them. After a while, they may get used to the DSL and overall Mathematica workflow enough to be able to use parts of Mathematica directly.

As to learning and using Mathematica directly, this is also possible IMO, but your company will probably have to allocate a lot of resources, either sending the employees to WRI courses, or creating some in-house training courses. This will also probably require a really high motivation from the average user. I've heard many statements of how easy it is to learn Mathematica. Sadly, I only find this partly true - there are many subtleties, from which a beginner is not isolated and which can take a while to digest. I think it also depends on how "far" are the majority of tasks from what is already available in Mathematica.

So, to summarize: my opinion is that Mathematica definitely has a potential to cut your costs and save your department lots of time, but you will either have to make an effective learning system at your department, or implement some framework or language which would, at least during the initial stage of adoption, shield the new users from the complexity of the full Mathematica language. My two cents.

Answer 3

I can’t speak directly about introducing Mathematica into engineering firms, but on the basis that many economists are wannabe engineers who couldn’t handle the maths, perhaps my observations will have some relevance. In my field, Mathematica is definitely a minority taste; Excel, eViews and Matlab dominate for different tasks.

Mathematica is not a data storage solution, whereas Excel is often used as one. Ideally, you should consider you data management strategy in conjunction with your choice of tools. People will be more comfortable adopting Mathematica if they are comfortable with where the end results are stored. You might need to write some convenience functions to get data in and out of databases or spreadsheets. If you use spreadsheets to store data (and I’m guessing that you do given the workflows you describe), you will need to specify some protocols on how those spreadsheets are set up. That way you will be able to automate some of those data access tasks more easily and robustly.

Different groups within the organisation will have different levels of sophistication and therefore need different tools. The Mathematica family of products is ideal for accommodating this: the maintainer of the custom functions and packages would do so using Wolfram Workbench; mainstream and sophisticated end-users would use vanilla Mathematica with the custom functions installed; management and less-sophisticated users can interact with CDF reports and applications in Player or Player Pro.

I endorse Leonid’s recommendation to create a Domain-Specific Language on top of Mathematica. I have even seen this done for APL (in another organisation, not where I currently work), so it can definitely be done for Mathematica. If you have someone who can maintain the underlying packages, and they are allowed to take the time to do so, then this can work fine. If the packages are in the Autoload directory, users don’t even need to be aware that they are using an add-on.

• At the risk of offending Infix and Postfix fans, in my experience, new users often find the “normal” syntax of function[arguments] to be more accessible; Map (/@) and Apply (@@) are usually the next bit of syntax they learn to handle. It will therefore help to design the user-facing functionality to use a more limited syntax, and hide things requiring replacement rules, Thread, Fold etc away behind custom functions. We used a simplified Mathematica syntax in the web front end for a database application, to define calculations. People actually warm to the idea that [] is for functions, {} is for lists, and () is for grouping.

• You will need to provide training, especially for any users who have experience with Matlab, Fortran or any other language that encourages loop-itis.

• You will also need to provide good documentation of any of the custom functions you create, but fortunately Mathematica allows that documentation to be integrated in the normal help in a relatively seamless way.

To encourage users to adopt the functionality, I would suggest leveraging Mathematica’s visualisation capabilities, including Manipulate. If you can create a “cool” Manipulate in a CDF that the management can use and need to look at (e.g. a key model or management report), you will get buy-in from the top - and that can be half the battle.