Simple fix. Put With[{i=i}]
to get the indexing in the dynamic right:
a = Table[List[
StringJoin[{"Number of neurons in layer ", ToString[i], ":"}],
With[{i = i}, InputField[Dynamic[d[i]], Number]]], {i, 3}];
AppendTo[a, {CancelButton[], DefaultButton[DialogReturn[]]}];
CreateDialog[Grid[a, Spacings -> {1, Automatic}, Alignment -> Left], Modal -> True];
On a sidenote, When I paste your code into Mathematica the i
in InputField[Dynamic[d[i]], Number]
turns red. If you highlight such a red character and go to the menu "help/why the coloring" it actually tells you that the order of evaluation conflicts.
I have added some more details on why this problem occurs, and why the solution works to a previous related question: Generating dynamic Inputfields