Refueling optimization of pressurized water reactor (PWR) cores is a key aspect of the safe and efficient operation of nuclear power plants. Traditional optimization methods often suffer from low computational efficiency and a tendency to fall into local optima. This paper proposes a refueling optimization method based on a combination of variational autoencoders, deep metric learning, and Bayesian optimization. The method utilizes variational autoencoders to map discrete core layout data into a continuous latent space, and deep metric learning is used to construct a structured latent space where samples with similar core parameters are placed close together. Then, a multi-objective Bayesian optimization method is employed to efficiently search for the optimal solution in the latent space, and the decoder is used to transform the optimal latent variables back into corresponding core layouts. Experimental validation based on M310 first-cycle initial loading data demonstrates that the proposed method can effectively improve refueling optimization efficiency and solution quality, yielding better refueling schemes than traditional methods.