Is there any good resource dealing with both Mathematica and bayesian inference?
I know of only one reference: the book by Phil Gregory "Bayesian Logical Data Analysis for the Physical Sciences". I can recommend this book. It covers a great deal of Bayesian data analysis and includes many worked examples.
It is published by Cambridge University Press: http://bit.ly/1tdaYhG . It is accompanied by a large(!) Mathematica notebook, which you can download from the Resources-tab at the same web page.