If you create a Task, and you don't ever call task.Wait() or try to retrieve the result of a Task<T>, when the task is collected by the garbage collector, it will tear down your application during finalization. For details, see MSDN's page on Exception Handling in the TPL.

The best option here is to "handle" the exception. This can be done via a continuation - you can attach a continuation to the task, and log/swallow/etc the exception that occurs. This provides a clean way to log task exceptions, and can be written as a simple extension method, ie:

public static void LogExceptions(this Task task)
{
    task.ContinueWith( t =>
    {
         var aggException = t.Exception.Flatten();
         foreach(var exception in aggException.InnerExceptions)
             LogException(exception);
    }, 
    TaskContinuationOptions.OnlyOnFaulted);
}

With the above, you can prevent any task from tearing down the app, and logging it, via:

Task.Factory.StartNew( () => 
   { 
       // Do your work...
   }).LogExceptions();

Alternatively, you can subscribe to the TaskScheduler.UnobservedTaskException and handle it there.