Simple fix. Put With[{i=i}] to get the indexing in the dynamic right:

a = Table[List[
StringJoin[{"Number of neurons in layer ", ToString[i], ":"}],
With[{i = i}, InputField[Dynamic[d[i]], Number]]], {i, 3}];
AppendTo[a, {CancelButton[], DefaultButton[DialogReturn[]]}];

CreateDialog[Grid[a, Spacings -> {1, Automatic}, Alignment -> Left], Modal -> True];

On a sidenote, When I paste your code into Mathematica the i in InputField[Dynamic[d[i]], Number] turns red. If you highlight such a red character and go to the menu "help/why the coloring" it actually tells you that the order of evaluation conflicts.

I have added some more details on why this problem occurs, and why the solution works to a previous related question: Generating dynamic Inputfields

Simple fix. Put With[{i=i}] to get the indexing in the dynamic right:

a = Table[List[
StringJoin[{"Number of neurons in layer ", ToString[i], ":"}],
With[{i = i}, InputField[Dynamic[d[i]], Number]]], {i, 3}];
AppendTo[a, {CancelButton[], DefaultButton[DialogReturn[]]}];

CreateDialog[Grid[a, Spacings -> {1, Automatic}, Alignment -> Left], Modal -> True];

On a sidenote, When I paste your code into Mathematica the i in InputField[Dynamic[d[i]], Number] turns red. If you highlight such a red character and go to the menu "help/why the coloring" it actually tells you that the order of evaluation conflicts.

I have added some more details on why this problem occurs, and why the solution works to a previous related question: Generating dynamic Inputfields

Simple fix. Put With[{i=i}] to get the indexing in the dynamic right:

a = Table[List[
StringJoin[{"Number of neurons in layer ", ToString[i], ":"}],
With[{i = i}, InputField[Dynamic[d[i]], Number]]], {i, 3}];
AppendTo[a, {CancelButton[], DefaultButton[DialogReturn[]]}];

CreateDialog[Grid[a, Spacings -> {1, Automatic}, Alignment -> Left], Modal -> True];

On a sidenote, When I paste your code into Mathematica the i in InputField[Dynamic[d[i]], Number] turns red. If you highlight such a red character and go to the menu "help/why the coloring" it actually tells you that the order of evaluation conflicts.

Simple fix. Put With[{i=i}] to get the indexing in the dynamic right:

a = Table[List[
StringJoin[{"Number of neurons in layer ", ToString[i], ":"}],
With[{i = i}, InputField[Dynamic[d[i]], Number]]], {i, 3}];
AppendTo[a, {CancelButton[], DefaultButton[DialogReturn[]]}];

CreateDialog[Grid[a, Spacings -> {1, Automatic}, Alignment -> Left], Modal -> True];

On a sidenote, When I paste your code into Mathematica the i in InputField[Dynamic[d[i]], Number] turns red. If you highlight such a red character and go to the menu "help/why the coloring" it actually tells you that the order of evaluation conflicts.

I have added some more details on why this problem occurs, and why the solution works to a previous related question: Generating dynamic Inputfields