We recommend setting at least 64MB of memory. But it mainly depends on plugins that you have or plan to have on your website, so you might need to set more if plugins require you to or if you start to get memory exhausted errors.
More info on how to configure memory limits in WordPress Codex.