You can specify ImageSize in inches. From docs > ImageSize> MoreInformation:

 Specifications for both width and height can be any of the following: 
     ...
     72di         di inches (before magnification)
     ... 

Examples:

Row[{Plot[{Sin[x], Cos[x]}, {x, -2 Pi, 2 Pi}, Frame -> True, 
AspectRatio -> 1/GoldenRatio, ImageSize -> 72 2], 
Plot[{Sin[x], Cos[x]}, {x, -2 Pi, 2 Pi}, Frame -> True, 
AspectRatio -> 1/GoldenRatio, ImageSize -> 72 3], 
Plot[{Sin[x], Cos[x]}, {x, -2 Pi, 2 Pi}, Frame -> True, 
AspectRatio -> 1/GoldenRatio, ImageSize -> 72 5]}]

You can specify ImageSize in inches. From docs > ImageSize> MoreInformation:

 Specifications for both width and height can be any of the following: 
     ...
     72di         di inches (before magnification)
     ... 

Examples:

Row[{Plot[{Sin[x], Cos[x]}, {x, -2 Pi, 2 Pi}, Frame -> True, 
AspectRatio -> 1/GoldenRatio, ImageSize -> 72 2], 
Plot[{Sin[x], Cos[x]}, {x, -2 Pi, 2 Pi}, Frame -> True, 
AspectRatio -> 1/GoldenRatio, ImageSize -> 72 3], 
Plot[{Sin[x], Cos[x]}, {x, -2 Pi, 2 Pi}, Frame -> True, 
AspectRatio -> 1/GoldenRatio, ImageSize -> 72 5]}]

EDIT: Per Mr.Wizard's suggestion, adding various Paddings in inches:

inches = 72; 
Panel@ Plot[{Sin[x], Cos[x]}, {x, -2 Pi, 2 Pi}, Frame -> True, 
PlotRangePadding -> {0.001 inches, 0.001 inches}, 
ImagePadding -> 0.3 inches, ImageMargins -> 0.3 inches, 
AspectRatio -> 1/GoldenRatio, ImageSize -> 5 inches]