If you have downloaded the app from Google Play™, you can find it again easily from your Google Play account... Once you're connected to Internet (WiFi, 3G/4G or Ethernet), go to Google Play™, then My Apps.
My Apps is divided in two: Installed Apps, the apps you currently installed on your device (letting you know the apps that require an update), and All, showing the history of apps linked with your Google Play™ account.
You can then find your app in that section. You can also use the Search button.
If the app was provided with your device, it won't be displayed in your Google Play™ account. See the following FAQ to reinstall it.
Note that deleting an app leads to a loss of data linked to that app. If it is a game, you lose any level reached unless your game data was synchronized with Google Play Games™ (this feature is not present in all games).