When we replaced our window with a stained glass one, I used what Heartland used. Closed cell foam tape. You can get it on Amazon. You do have to remove the window and while doing so should have another person to hold it in place so it does not fall when one supporting side is removed. If I could do it, you can.
Foam Seal Tape,Weather Stripping for Door and Window Insulation,2 Rolls 1/4in Wide x 1/8 T Seal Strip,Closed Cell Single Sided Adhesive Window Seal,Total 32FT $8.99