I’m not sure how the structure of WP and Blogger sites are for their respective search engine optimized URLs, but if they are the same it should not be an issue. Google and other search engines will only see that your site IP address changed, but if the main URL and page URLs are the same then they will not penalize you.

In some cases, it might give you a teeny bump in rankings — say, if you went from shared wordpress.com with a shared IP address to your own hosting service with a dedicated IP address.