Fixed compile options polluting project (#694)

Fixed compile options polluting project (#694)

* Fixed compile options polluting project

Moved add_library higher

* Apply suggestions from code review

Co-authored-by: Paul Bartell <paul.bartell@gmail.com>
* fixed cmakelists keeping in mind the suggestions

---------

Co-authored-by: Paul Bartell <paul.bartell@gmail.com>
Co-authored-by: Gaurav-Aggarwal-AWS <33462878+aggarg@users.noreply.github.com>
Co-authored-by: Aniruddha Kanhere <60444055+AniruddhaKanhere@users.noreply.github.com>