Suppose I have a vector in $\mathbb{R}^n$ but $n$ is not known in advance. I want to be able to write functions which operate on the components of that vector, and then I'd like to be able to take derivatives with respect to the components. As an example, consider the relation: $$\frac{\partial}{\partial x_j} \sum_i x_i$$
Under the assumption that the $x_i$ are independent, I want a call to Simplify[] to return 1. Similarly, calling Simplify[] on $\frac{\partial x_i}{\partial x_j}$ should give KroneckerDelta[i,j]. It's not clear how I should represent generic coordinates like this. I've seen this, but I'm not sure it provides an answer. As the linked post suggests, I could do this for a fixed $n$, but that's not situation I'm working on, especially since I want to see the generic form for any $n$.
For reference, it seems that sympy let's you do something close to this.
from sympy.tensor import IndexedBase, Idx
x = IndexBase('x')
i, j = map(Idx, ['i', 'j'])
x[i]
x[i].diff
# <bound method Indexed.diff of x[i]>
x[i].diff(x[j])
# ValueError: Can't differentiate wrt the variable: x[j], 1
But despite being able to represent the variables abstractly, I can't seem to differentiate them.
Here is another example. Suppose you wanted to calculate the derivative of the entropy with respect to one of the components of the input distribution (again, assuming all the variables are independent). The final form is the same no matter what $n$-simplex the distribution lives on, so you'd like to be able to do this for any dimension.
$$ \frac{\partial H}{\partial p_j} = - \frac{\partial}{\partial p_j} \sum_i p_i \log p_i = - (\log p_j + 1)$$
Problems like this come up in optimization, when you need to provide the gradient and Hessian to numerical algorithms.
Update: Here are two other related posts:
