\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[a4paper, total={6in, 8in}]{geometry}
\usepackage{amsmath, amsfonts}
\usepackage{hyperref, graphicx}

\title{CSCI 145 Problem 16: What Does an Embedding Identify?}
\author{} % TODO: Put your name here
\date{\today}

\begin{document}
\maketitle

\subsection*{Submission Instructions}
Please upload your work to Gradescope by \textbf{11:59pm on DATE}. % TODO: date
\begin{itemize}
\item Turn in a single PDF of \textit{handwritten} solutions. Typed (including \LaTeX) solutions will not be accepted.
\item Problems are grouped into three-week \textbf{cycles}. Once per cycle, you will meet with the grader and present the solution to \emph{one} problem from that cycle, selected at random when you arrive; bring your work and be ready to explain any of them.
\end{itemize}

\newpage \section*{Problem 16: What Does an Embedding Identify? \hfill \textnormal{2 points}}

A linear autoencoder represents a data matrix $\mathbf{X}$ by codes $\mathbf{Z}=\mathbf{X}\mathbf{W}_0\in\mathbb R^{n\times r}$ and reconstruction $\mathbf{Z}\mathbf{W}_1$. A lab reports that coordinate 7 of its trained code is a meaningful scientific variable. Its reconstruction error is excellent, and a contrastive model trained on the same data gives similar nearest neighbors.

Determine exactly which changes of latent coordinates these observations can and cannot rule out. For an arbitrary invertible $\mathbf{R}\in\mathbb R^{r\times r}$, identify a corresponding change to $\mathbf{W}_0,\mathbf{W}_1$ that leaves every reconstruction unchanged. Characterize, with proofs, the matrices $\mathbf{R}$ that preserve for every pair of latent vectors in $\mathbb R^r$:
$$
\text{all dot products},\qquad
\text{all Euclidean distances}.
$$

Use your characterization to decide what evidence the lab would need before assigning an interpretation to one coordinate rather than to the geometry of the representation as a whole.

\paragraph{Optional starred extension $(\star)$.}
This extension is not required for full credit. For $r\geq2$, characterize all invertible $\mathbf{R}$ that preserve every cosine similarity. Then determine whether a contrastive objective whose logits are dot products divided by a learned temperature can distinguish every transformation in this larger class.

\paragraph{Results you may use.}
You may use the polarization identity and the fact that $\mathbf{R}$ preserves every Euclidean norm exactly when $\mathbf{R}^\top \mathbf{R}=\mathbf{I}$.

%\input{solutions/solution16}
\end{document}
