If you can't remove public folder database on your old Exchange 2003 server (and after having replicate correctly data to Exchange 2010)
You can try this :
ADSI Edit >
+ CN=Microsoft Exchange
+ CN=YOUR ORGANISATION
+ CN=Administrative Groups
+ CN=First Administrative Group
+ CN=SERVER NAME
Delete manually and dismounted the store.
You are able to uninstall Exchange 2003.