What are the requirements for a well behaved indexed variable? Subscript, ToExpression, Downvalue?

On writing this answer I needed to call a function, (NonlinearModelFit) with an unknown number of parameters.

We have learned that we should not use Subscript for indexing variables because statements such as are actually an assignment to Subscript not to x.

So my solution was to construct a String and then use ToExpression

kvar[k_Integer] :=
ToExpression@
Map[StringJoin[#, ToString[k]] &, {"x", "σ", "a"}]


Giving

kvar[3]


{x3, σ3, a3}

That is nice as each variable is an actual AtomQ and Symbol , but generating them from Strings seems not elegant to me.

Another solution would have been to use DownValues

kvar[k_Integer] := Through[{x, σ, a}[k]]


Giving

kvar[3]


{x[3], σ[3], a[3]}

Which is not a Symbol nor ?AtomQ, yet it works just fine for that task in hand. I'm unsure of when this solution could become a problem.

I'm aware of the existence of Notation and Symbolize, but I'm not sure if that is a nice "good practices" solution.

My questions are:

What is the recommended and most elegant form of indexed variables?

What are the requirements for well behaved variables?

Is it ever relevant if the Head is Symbol or if its ?AtomQ?

• Have you looked at Indexed? Sep 9 '15 at 13:50
• @rcollyer Indexed[x, 3] = 4 gives Set::write: Tag Indexed in Indexed[x,3] is Protected. >> How do you propose to use it as indexed variables? Sep 9 '15 at 14:00
• Sorry. It's intent is slightly different than what you asked for. It represents a symbolic indexed variable, so it would not have a value, itself. I was wondering if its use would serve your purposes, per its docs. Sep 9 '15 at 14:09
• We have learned that we should not use Subscript for indexing variables ... You shouldn't have any problem, provided you don't mix subscripted $x_2$ and non-subscripted $x$ at the same time. Jul 15 '16 at 4:09

General usage

Here is what I think

• Using strings and subsequently ToString - ToExpression just to generate variable names is pretty much unacceptable, or at the very least should be the last thing you try. I don't know of a single case where this couldn't be replaced with a better solution
• Using subscripts is also pretty bad and should be avoided, except for purely presentation purposes - as you noted
• For cases when you need to use many generated variables, indexed variables are usually the best way to go. They usually take the form

head[index]


and can be used im most places where usual variables can be used, particularly in equations or other expressions of symbolic (inert) nature. You need a bit more care with indexed variables, than plain symbols, in particular it is best to ensure that the index is either numeric or, if an expression, should be inert in the sense of evaluation (keep the same value always, or no value).

• Sometimes, you can also use the symbols generated by using Unique[...]. Usually, they are used as temporary anonymous placeholders in some intermediate transformations, but then you will have to make sure they are destroyed after you no longer need them.

Assignments and state

A very important aspect here is whether the variables are intended to be inert symbolic entities, or you plan to store some values in them. Here are a few things to keep in mind:

• Values stored in variables will be stored in different types of rules for symbol variables and indexed variables:

• For symbol-based variables, these will be in OwnValues
• For indexed variables, these will be in DownValues, or sometimes SubValues, if you use nested indices.
• Only symbols allow part assignments. So, for example, you can do

a = Range[10];
a[[5]] = 100;


but you can't do

a[1]=Range[10]; (* Ok by itself *)
a[1][[5]] = 100 (* Won't work *)


This can be a big deal, for some applications

• Only symbols can serve as local variables / constants in Module, Block, With, Function, Pattern, etc.

• For the case of many variables, indexed variables may be easier to manage, since you have to clear only one symbol.

• To selectively clear a given indexed variable, you have to use Unset, not Clear:

 a[1]=.

• Indexed variables can not be used inside Compile, although it may appear that they can.

• If you must do assignments to many (indexed) variables, I'd consider using an Association instead. This may make it easier from the resource management point of view, since you can store an association in a single variable. An additional bonus is that then, part assignments to particular indexed variables are allowed:

assoc = <|a -> {1, 2, 3}, b -> {4, 5, 6}|>;
assoc[[Key[a], 2]] = 10;
assoc

(* <|a -> {1, 10, 3}, b -> {4, 5, 6}|> *)


Notes

As far as I can recall now, being AtomQ is not a requirement for most uses for variables. Being a plain Symbol is required in some cases, like for local variables in scoping constructs, or part assignments - as I explained above.

In general, my experience is that most of uses for indexed variables in pure programming context are more or less equivalent to using a hash table. In the context of symbolic manipulations, indexed variables can be quite useful in many ways - they can represent, for example, coefficients for powers in a polynomial, and many other things.

For anything involving programming / transformations, I'd stay away from Subscript, Notation, Symbolize, and all other things that can mix evaluation and presentation aspects. Using them in code is just an invitation for trouble. If you want to format an expression in some way, write special functions which would do that, as a separate stage.

• There is at least one exception I know: With[{x[1] = 3}, x[1] + x[1] ] and I was sure I have seen some built-in function like Integrate complaining and throwing mysterious errors when using indexed variables, but I cannot recall what exactly the problem was. Then you are stuck with one of the not so favorable solutions. Sep 9 '15 at 14:25
• @halirutan The With example you gave doesn't work for me, gives an error. For Integrate, in principle indexed variables should work - if they don't in some cases, I'd say it is a bug. Sep 9 '15 at 14:27
• @rhermans Unique works also with no arguments or with a symbol argument, so is not tied to strings. As to ToString - ToExpression cycles, there are several reasons to avoid them. The general one is that they lead us outside of the symbolic world, and involve parsing, which is bad both for performance reasons and also just because one should avoid parsing as much as possible (for example, it may be unsafe). Sep 9 '15 at 15:33
• I think one of the main disadvantages of using ToExpression for such purposes is that your code typically will break when putting it into a package (or make use of namespaces (Contexts) in any way). The problem is that ToExpression generates the new symbols in the current $Context, whatever that happens to be -- or a symbol with the same name somewhere in the current $ContextPath will be "abused". My experience is that 90% of the problems when turning code into a decent package comes from ToExpression (or Symbol). Sep 9 '15 at 16:36
• @Mr.Wizard I just meant that this needs to be reduced to only the cases where this is really needed. Just using ToExpression (or Symbol) is fine when that is really necessary. But ToString followed by ToExpression means that you basically leave the symbolic world for strings and then return back, via parsing / symbol construction. In lots of cases, this is not really necessary and can be replaced with something that would not require the serialization / parsing stage. That's all I meant here, really. Feb 7 '17 at 16:53

Using DownValues enables you to format the display in the subscripted form without using Notation and Symbolize

(Format[#[n_]] := Subscript[#, n]) & /@ {x, σ, a};

kvar[k_] := Through[{x, σ, a}[k]]

kvar[3]


kvar[n]


If you will never use a symbolic index then you can restrict the argument of kvar to Integer as you did originally.

What are the requirements for well behaved variables?

Functions are not variables, although in most cases, the kernel treats undefined variables and functions identically. Sometimes it doesn't. After all, there are places in mathematics where the difference between a number and a function is important.

One extreme and undocumented example is Dt[], the total derivative function. There,f[1] is very much different from f1. f[1] is a number, the value of f at 1, constant by definition of a mathematical function of one variable, while f1 is not assumed to be constant unless an explicit declaration is made.

f[1]          f[1]

AtomQ[f[1]]   False

Dt[f[x], x]   f'[x]

Dt[f[y], x]   Dt[y, x] f'[y]

Dt[f[1], x]   0

Dt[f1, x]     Dt[f1, x]

D[f[1], x]    0


What is the recommended and most elegant form of indexed variables?

A very simple method is to make this definition for each indexed variable:

x[i_Integer] := x[i] = With[{u = Unique[x]}, Format[u] = Subscript[x, i]; u]


Or,

 defineIndexedVariable[x_Symbol] := (
x[i_Integer] := x[i] = With[{u = Unique[x]}, Format[u] = Subscript[x, i]; u]
)


It allows negative subscripts, doesn't use Symbol, and Unique[x] handles Context properly, but InputForm[Array[x,5]] prints (say) {x$2136, x$2152, x$2163, x$2164, x$2165}. There is no temptation to sometimes write x[1] and sometimes write x$2136.

Another more complicated method of constructing symbols on the fly is well, not very elegant, and is an example of how complicated things can suddenly collapse into simplicity, but it does avoid repeated string operations and problems with $Context. It allows us to write v[1] and have it print as Subscript[v, 1] (which displays$\text{v}_1$), and have {InputForm[v[1]], Head[v[1]], AtomQ[v[1]]} evaluate to {v⎵1, Symbol, True}. InputForm[Array[v,5]] prints {v⎵1, v⎵2, v⎵3, v⎵4, v⎵5}, and we can sometimes write v[1] and sometimes v⎵1. Simply paste this function into your notebook and evaluate it for each indexed symbol. Each nonnegative integer subscript will evaluate the symbol's function exactly once. The first version evaluates Context[FUN]<>SymbolName[FUN] every time a new subscript is encountered. defineIndexedVariable[FUN_Symbol] := FUN[ix_Integer /; ix ≥ 0] := With[{ v=Symbol[Context[FUN]<>SymbolName[FUN]<>"⎵"<>ToString[ix]] }, Format[v] = Subscript[FUN,ix]; FUN[ix] = v ]  For one symbol, optimize it by hand and specify context explicitly everywhere: FUNFUN[ix_Integer /; ix ≥ 0] := With[{ v=Symbol["FUNFUN⎵"<>ToString[ix]] }, Format[v] = Subscript[FUNFUN,ix]; FUNFUN[ix] = v ]  Or get the same optimization with this version: defineIndexedVariable[FUN_Symbol] := With[{ prefix = Context[FUN] <> SymbolName[FUN] <> "⎵" }, FUN[ix_Integer /; ix >= 0] := With[{ v = Symbol[prefix <> ToString[ix]] }, Format[v] = Subscript[FUN, ix]; FUN[ix] = v ] ]  The Context[FUN] prefix ensures that all of the new symbols will be in the same context as the function definition. The FUN[ix] = memoizes the function so Symbol is called only once for each distinct index, and the Format[v] definition is made only once. The absence of a semicolon after the FUN[ix] = v is absolutely essential. After defineIndexedVariable[v], we get (OutputForm on a text terminal)  v v[1] 1 Head[v[1]] Symbol AtomQ[v[1]] True Dt[v[x], x] v'[x] Dt[v[y], x] Dt[y, x] v'[y] Dt[v , x] Dt[v[1], x] 1 Dt[v1, x] Dt[v1, x] D[v[1], x] 0  Information["v"]: Globalv v[1] = v⎵1 v[ix$_Integer/;ix$>=0]:=With[{v$=Symbol["Globalv⎵"<>ToString[ix$]]},Format[v$]=Subscript[v,ix$];v[ix$]=v$]  Using the standard front end, ??v does not showv⎵1. But InputForm[Array[x,5]] prints {v⎵1, v⎵2, v⎵3, v⎵4, v⎵5} Code that printed the above tables: Function[x,{Table[" ",{Length[x]}],HoldForm/@Unevaluated[x],x}, HoldAll][{v[1],InputForm[v[1]],Head[v[1]],AtomQ[v[1]],Dt[v[x],x],Dt[v[y],x],Dt[v[1],x],Dt[v1,x],D[v[1],x]}]//Transpose//TableForm//Print  • I'd like to point out a limitation of this method: it still requires the indexed variable to evaluate to a symbol so it'll almost always have trouble when evaluation order matters. For example, you still can't define Function[x[1], x[1]] after defineIndexedVariable[v]. Aug 4 '17 at 7:03 A kind of recommended form for the indexed variables would be to use Symbolize function from the Notation package: Get["Notation"] Symbolize[ParsedBoxWrapper[$$a \_ 1$$]]  after that you can treat a with superscript 1 as a usual symbol. In particular AtomQ will give True; you can assign a value to it using Set; and Clear[Subscript] will not clear your definitions. Also see FullForm to convince yourself that it is now a solid symbol. Note: it will only work if your output settings are Standard • I would argue that "most recommended" is limited to cases where presentation is more important than code reliability and reusability... Sep 9 '15 at 16:38 • This is quite reliable. Even though I personally prefer 1D notations. This could considerably increase readability of the code. Sep 9 '15 at 16:48 • Well, I think it depends on the frontend and the functionality to turn boxes into expressions. Messing with these makes me feel uncomfortable as it makes code too difficult to understand for me, especially when errors occur. But true, it might well work reliable if used correctly and in the scope it is meant to... Sep 9 '15 at 16:54 • That's what I tried to express: there is much going on under the hood and things become more complicated than they need to. My experience is that code doesn't become more reliable if it is more complicated then necessary. I think it is absolutely OK to use the Notation package if you have need for it but just wouldn't call it "most recommended" for all use cases... Sep 9 '15 at 22:48 • You just type expression and on the way to the kernel it gets replaced by this monster and then back on the way back. The same as with any other 2D expression. Anyway I edited "Most recommended" as I personally would rather do a[1] just because that's faster. Sep 9 '15 at 23:42 A package supporting fairly elegant subscripted symbols, where a subscript can be any invariant atom. In[2]:= newSubscriptedSymbol[x] Out[2]= x In[3]:= x[1] Out[3]= x 1 In[4]:= newSubscriptedSymbol[y] Out[4]= y In[5]:= x[y[Pi]] Out[5]= x y Pi In[6]:= subscriptInformation[x] x x[1] 1 x$142

x
x[y  ]    y
Pi      Pi   x$144 In[7]:= subscriptInformation[y] y y[Pi] Pi y$143

In[8]:= ??x$144 Globalx$144

Attributes[x$144] = {Protected} Format[x$144] = Subscript[x, y$143] MakeBoxes[x$144, FormatType_] ^= Format[Subscript[x, y$143], FormatType] In[9]:= ??y$143
Globaly$143 Attributes[y$143] = {Protected}

Format[y$143] = Subscript[y, Pi] MakeBoxes[y$143, FormatType_] ^= Format[Subscript[y, Pi], FormatType]

In[10]:=


The above from a text terminal in OutputForm. Also tested in the front end, where it looks nicer.

BeginPackage["newSubscriptedSymbol"]

newSubscriptedSymbol::usage = "newSubscriptedSymbol[x] defines x[atom] to return a unique protected symbol for each unique atom, that formats as Subscript[x,atom].\n\n"<>
"subscriptInformation[x] prints a table of the subscript atom, the formatted symbol, and the unformatted unique symbol."

subscriptInformation::usage = "subscriptInformation[newSubscriptedSymbol[x]] prints a table of the subscript atom, the formatted symbol, and the unformatted unique symbol."

Begin["Private"]

newSubscriptedSymbol[x_Symbol] := (
newSubscriptedSymbol[x] ^= x;
subscriptInformation[x] := Print[TableForm[Cases[DownValues[x],Verbatim[RuleDelayed][Verbatim[HoldPattern][Verbatim[x][a_]],s_Symbol]->{HoldForm[x[a]], s, InputForm[s]},{1}]]];
x::notprotected = "newSubscriptedSymbol["<>SymbolName[x]<>"][]:  is a symbol, but it is not protected.";
x[sym_Symbol /; Not[MemberQ[Attributes[sym],Protected]] ] := (Message[x::notprotected,sym,sym]; $Failed); x[atom_?AtomQ] := With[{u=Unique[x]}, Format[u] = Subscript[x,atom]; Unprotect[x]; x[atom] = u; Protect[x]; Protect[u]; u ]; x::usage = SymbolName[x]<>"[atom] returns a unique protected symbol which formats as Subscript["<>SymbolName[x]<>",atom].\n\n"<> "subscriptInformation["<>SymbolName[x]<>"] prints a table of the subscript atom, the formatted symbol, and the unformatted unique symbol."; x[___] := (Message[x::usage];$Failed);
`