I see you have got the correct answer -- just to point out it appears the distribution is actually properly discrete, its just PDF[] that is not returning the correct discrete form:

ListPlot[#/{1, 100} & /@ 
      Tally@Table[ 
      RandomVariate[
      ProbabilityDistribution[1/4, {x, 0, 3, 1}]] , {100}], 
         PlotRange -> {0, 1/2}, Filling -> Axis, PlotStyle -> PointSize[.02], 
         Epilog -> {Line[{{0, 1/4}, {3, 1/4}}]}]

I see you have got the correct answer -- just to point out it appears the distribution is actually properly discrete, its just PDF[] that is not returning the correct discrete form:

ListPlot[#/{1, 100} & /@ 
      Tally@Table[ 
      RandomVariate[
      ProbabilityDistribution[1/4, {x, 0, 3, 1}]] , {100}], 
         PlotRange -> {0, 1/2}, Filling -> Axis, PlotStyle -> PointSize[.02], 
         Epilog -> {Line[{{0, 1/4}, {3, 1/4}}]}]

I won't use the "b" word but this strikes me as something that should be fixed -- nothing in the docs indicates that you should need to supply a discrete function to ProbabilityDistribution to use its discrete form.