
% http://css.paperplaza.net/conferences/support/files/IEEEtran_HOWTO.pdf
% http://tex.aspcode.net/view/635399273629833626234768/problem-having-copyright-notice-and-header-at-the-same-time-in-ieeetrans-template

% http://www.ieee-lascas.org/lascas2015/
% LASCAS 2015 - Montevideo (B5)
% Paper Submission: November 3rd, 2014.
% Notification of acceptance: December 8th, 2014.
% Camera-ready:	December 23rd, 2014.

\documentclass[conference, 10pt, twocolumn]{IEEEtran}
\IEEEoverridecommandlockouts

\usepackage[utf8]{inputenc} % accents
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{paralist}     % inline lists
\usepackage{booktabs}     % better tables
\usepackage{colortbl}     % colors in tables
\usepackage{xcolor}       % colors (names)
\usepackage{nicefrac}     % better (smaller) inline fractions...
\usepackage{cite}         % merge citations with hyphens, and sort they
\usepackage{balance}      % balance last page columns
\usepackage{gensymb}      % degree symbol
\usepackage{textcomp}     % micro symbol
\usepackage{fixltx2e}     % ERROR with IEEEpubid =/
\usepackage{amsmath}      % plusminus \pm symbol
\usepackage[hyphens]{url}


% Signal command to reference signals and items in figures.
\newcommand{\signal}[1]{{\footnotesize{\textsf{#1}}\normalsize}}

% \setlength{\parskip}{0.2\baselineskip plus2pt minus2pt}
% \setlength{\parskip}{0.4\baselineskip plus2pt minus2pt}
\linespread{0.97} % original linespread is 1

% Reduce a little the space before equation
\setlength{\abovedisplayskip}{0pt} 
\setlength{\abovedisplayshortskip}{0pt}

% set space after table
\setlength{\textfloatsep}{0.4cm}

\begin{document}%

\title{A Digitally Controlled Oscillator for Fine-Grained\\ Local Clock Generators in MPSoCs}

\author{\large
Guilherme Heck, 
Leandro S. Heck,
Matheus T. Moreira, \\
Fernando G. Moraes and 
Ney L. V. Calazans \\
Pontifícia Universidade Católica do Rio Grande do Sul - Porto Alegre, Brazil \\
\{guilherme.heck, leandro.heck, matheus.moreira\}@acad.pucrs.br, \\
\{fernando.moraes, ney.calazans\}@pucrs.br
}


% HEADING - DONT TOUCH THIS
% \markboth{2015 IEEE 6th Latin American Symposium on Circuits & Systems (LASCAS)}{}

% LH: COPYRIGHT - DONT TOUCH THIS !!!
%  Copyright notice is: 978-1-4799-8332-2/15/$31.00 ©2015 IEEE
\IEEEoverridecommandlockouts
\IEEEpubid{\makebox[\columnwidth]{978-1-4799-8332-2/15/\$31.00~\copyright~2015 IEEE \hfill} \hspace{\columnsep}\makebox[\columnwidth]{ }}
\IEEEpubidadjcol
\IEEEpubidadjcol


% make the title area
\maketitle
% \thispagestyle{empty}\pagestyle{empty}

%#
%# ABSTRACT 
%#=========
\begin{abstract}
\boldmath
The evolution of technology into deep submicron domains leads to increasingly complex timing closure problems to design multiprocessor systems. 
One natural alternative is to resort to the globally asynchronous, locally synchronous paradigm (GALS). 
This work proposes a generic architecture for very low power- and area-overhead local clock generators (LCG) to drive individual modules of a multiprocessor, \emph{e.g.} network on chip routers and other elements. 
As main original contribution it details the design of a digitally controlled oscillator (DCO), the core of the clock generator architecture. 
This DCO can produce at least 16 distinct frequencies between 117~MHz and 1~GHz and supports clock gating and glitch-free frequency changes. 
Its design is robust to PVT variations and takes less than 1000~\textmu m\textsuperscript{2}.
\end{abstract}

\begin{IEEEkeywords}
Local clock generator, LCG, PVT, MPSoCs.
\end{IEEEkeywords}



%#
%# SECTION: INTRODUCTION
%#=======================
\section{Introduction and Related Work}
\label{sec:intro}

Network-on-chip (NoC) based multiprocessor systems-on-chip (MPSoCs) can deal with semiconductor market evolving requirements, which has for many years capitalized on the synchronous paradigm. 
The latter assumes the use of a single clock signal to control the design sequencing of events, which drastically reduces complexity. 
However, in modern technologies the assumption of global synchronicity in MPSoCs leads to problems in timing closure, as process variability stresses design constraints to unmanageable levels.

Today, the globally asynchronous, locally synchronous (GALS) design paradigm \cite{Ch84} is a practical design option for MPSoCs, allowing the use of synchronous elements that communicate asynchronously at system level. 
But in such a system, there are at least two new problems to solve: system partitioning into frequency domains and the choice of a clock generation scheme. 
This work addresses the second problem. It proposes a generic architecture for local clock generators (LCGs) targeted to GALS NoC-based MPSoCs. 
The architecture focus on low power and area, to minimize overheads. 
It enables the use of up to one distinct clock generator per MPSoC module, either an intellectual property core (IP) or an NoC router.

As main original contribution the paper details the design of a digitally controlled oscillator (DCO) for the architecture. The DCO can produce any of 16 distinct operating frequencies between 117~MHz and 1~GHz, with process, voltage and temperature (PVT) compensation capability, clock gating and glitch-free frequency switching support. 
Traditional techniques to design clock generators employ PLLs or similar structures, to achieve highly precise phase and frequency, leading to high costs in power and area. 
Our approach trades off precision, power and area. External controllers can add fidelity if needed.

Although the ITRS \cite{ITRS11} states that intrachip asynchronous global signaling will be increasingly needed, few works address DCOs or LCGs for MPSoCs. 
In \cite{SLP08}, authors propose a standard cell-based DCO with clock gating support using inverters and NANDs. 
Results show a 22\% uncertainty in the generated frequency but process variations can produce up to 70\% deviations. 
References \cite{YCZ12} and \cite{RCM+12}, propose the suppression of clock pulses to reduce a base frequency. 
However, the maximum supported operating frequency determines timing constraints that must be respected globally, which may be unreasonable for MPSoCs. 
Albea et al. \cite{APVP+11} propose an LCG for DVFS on GALS MPSoCs which uses a DCO with R-2R binary coded networks for DACs.
It can produce 256 different clock values, but R-2R DACs are based on voltage references and on DVS circuits, being this dependence undesirable. 
The gap between corner cases can achieve around 3.5~GHz on a given frequency. 
Besides, the approach does not support clock gating. 
Höppner et al. \cite{HEH+13} propose a clock generator that produces 83 distinct frequencies between 83~MHz and 666~MHz for IPs and 2~GHz and 4~GHz frequencies for NoC routers. 
Although it allows different frequencies for IPs and routers, the circuit has high cost in complexity, area and power.


%#
%# SECTION 
%#==================
\section{The LCG Architectural Assumptions}
\label{sec:arch}

The LCG proposed herein addresses MPSoCs like those discussed in \cite{OMM+09, COCM09}. 
Fig.~\ref{fig:arch} presents the target MPSoC structure.
A router \signal{R} and an \signal{IP} compose a processing element \signal{PE}.
Routers and/or IPs have an operating frequency provided by the low power-, low area-overhead LCG that enables using fine-grained clock domains. 
Here, all LCGs employ an external low frequency reference clock (\signal{ref\_clk}) to provide overall control of the locally generated frequencies. 
The \signal{ref\_clk} signal drives the LCGs only, consuming negligible power and requiring no skew control, as signal phase is irrelevant. Routers and IPs require synchronization interfaces \cite{G11} in all interconnection points, \emph{e.g.} all ports of a router. 
Other clock schemes are also allowed, where groups of routers and/or IPs are in a same frequency island. 
This relaxes synchronizing interface requirements, but may prevent design space exploration of fine-grain frequency domains, and is application specific.

\begin{figure}[!t]
\centering
\includegraphics[width=0.8\linewidth]{figures/generic-mpsoc.pdf}
\caption{Block diagram for the target MPSoC structure and detailed view of a processing element PE. LCGs are the local clock generators.}
\label{fig:arch}
\end{figure}

%#
%# SECTION 
%#==================
\section{The Proposed Local Clock Generator}
\label{sec:proposed}

\begin{figure}[!t]
\centering
\includegraphics[width=0.75\linewidth]{figures/generic-lcg.pdf}
\caption{The proposed LCG architecture.}
\label{fig:lcg}
\end{figure}

\begin{figure}[!t]
\centering
\includegraphics[width=0.75\linewidth]{figures/dco.pdf}
\caption{Block diagram and block contents for the DCO.}
\label{fig:dco}
\end{figure}

% LH: Add a reference to justify the frequency range

The LCG structure comprises a \signal{Controller} and a \signal{DCO}, which in turn comprises an \signal{Actuator} and an \signal{Oscillator}. 
Fig.~\ref{fig:lcg} shows its architecture, which generates the adjustable clock (\signal{clk}). 
The \signal{ref\_clk} signal furnishes a time base for the generation process, while signals \signal{freq\_sel} and \signal{clk\_pause} respective allow choosing the clock frequency and requesting the clock to pause. 
The \signal{Controller} produces a set of binary control signals, generically designated \signal{ctrl} in the Figure.

The \signal{Actuator} receives part of \signal{ctrl} and a current source \signal{i\textsubscript{REF}} output, a reference to produce the main frequency control signal. 
This module compensates PVT variation effects. 
The external \signal{Controller} receives as feedback the generated frequency, and verifies if it corresponds to the expected value. 
If not, it reconfigures the DCO to reach the requested operating frequency. 
Typically, NoC routers do not need a precise clock frequency to operate; their usual role is to deliver packets in a rate close to that of the IP injecting data in it. 
Each router can individually select the best available frequency to operate in. 
Thus, the \signal{Controller} for routers LCGs can be extremely simple, reducing power and area overheads. 
The \signal{Controller} for IPs LCGs is typically more complex, to improve precision. 
This paper will not focus the design of the \signal{Controller}, only on the DCO design.


%#
%# SECTION 
%#==================
\section{The DCO Design}
\label{sec:design}

The LCG output frequency range is set between 117~MHz to 1~GHz, to cover typical operating frequencies of embedded systems.
Coupled to an assumption to use 16 frequency values, this determines frequency steps of around 59~MHz, considering the ideal use of a constant difference between any two successive frequency steps. 
The number of distinct frequencies resulted as the maximum value in an analysis of the noise and supply voltage margins offered by the chosen technology (65~nm). 
This assumes the use of minimum-size transistors in the DCO, to reduce the area overhead. To enable significant reductions in power when idle, the DCO embeds a clock gating capability to stop the oscillation. 

Fig.~\ref{fig:dco} depicts the proposed DCO structure. 
A set of external signals commands the frequency generation process. 
These correspond to the decomposition of the \signal{ctrl} control bus of Fig.~\ref{fig:lcg} in five components: \signal{comp}, \signal{fsel}, \signal{clk\_dis}, \signal{clk\_retain} and \signal{reset}. 
Signal \signal{comp} acts on the reference current source to compensate PVT variations; \signal{fsel} selects the DCO frequency after PVT calibration, helping  in the generation of the current that commands the \signal{Oscillator}, \signal{i\textsubscript{CTRL}}. 
Clock gating is achievable in two different ways: \begin{inparaenum}[\itshape i\upshape)] 
\item \label{itm:a1} activating \signal{clk\_dis} freezes both the feedback clock (\signal{fdbk\_clk}) and the clock (\signal{clk}) when these are 0, opening the oscillation ring;
\item \label{itm:a2} activating \signal{clk\_retain} holds just the clock signal level (\signal{clk}), to allow the Controller to adjust the DCO without interfering in the driven circuit. 
\end{inparaenum}

The DCO produces a clock signal from its digital inputs, which control the delay of a ring oscillator. It contains: 
\begin{inparaenum}[\itshape i\upshape)]
\item \label{itm:p1} a DAC to control PVT variations (\signal{PCDAC}); 
\item \label{itm:p2} a DAC to handle frequency selection requests (\signal{FSDAC}); 
\item \label{itm:p3} a current-controlled oscillator (\signal{ICO}), to produce the output frequency; and
\item \label{itm:p4} a clock retention circuit (\signal{CR}), 
\end{inparaenum}
to disable clock propagation to external entities, when requested. 
The following Sections detail the design of the \signal{Oscillator}, the \signal{Actuator} and their internal components.



%#
%# SUB-SECTION 
%#==================
\subsection{The Oscillator}
\label{subsec:oscillator}

\begin{figure}[!t]
\centering
\includegraphics[width=0.85\linewidth]{figures/rosc.pdf}
\caption{Circuit diagram of the ICO. The current mirror feeds the current-starved inverters (CSI). The \signal{Clock Gating Circuit} shares the \signal{I5} inverter with the oscillator.}
\label{fig:rosc}
\end{figure}

Fig.~\ref{fig:rosc} details the structure of the proposed oscillator (ICO). 
It relies on a ring formed by current-starved inverters (CSIs) \cite{Ba11}. 
Besides providing small area footprint, this structure allows manipulating oscillation time constants of the ring elements by using current instead of voltage control, which improves stability \cite{KMPHH08}. 
Given the power supply, the transistor nominal current and input capacitance for the technology, it is possible to determine the appropriate number of delay stages for the selected frequency constraints. 
In our case, five delay stages were enough to achieve the required frequency range.

One original contribution added to the well-known current-starved architecture appears in the fifth stage: a glitch-free \signal{Clock Gating Circuit}. 
An energy-efficient way to stop a clock consists in opening the ring. 
To avoid stability problems, modifications on the last stage of the oscillator allow it to operate under clock gating as a latch, using a pair of transmission gates, as Fig.~\ref{fig:rosc} shows.
The ring opens when \signal{TG1} opens and \signal{TG2} closes. The memory structure formed by inverters \signal{I5}, \signal{I6} and \signal{TG2} holds the output logic level. 
The NAND gate freezes the output clock signal in 0 without glitches, when asserting \signal{clk\_dis}, \signal{na} and \signal{nb}. Note that \signal{I6} is a high-drive inverter. 
If \signal{na} starts going to 0 when signal \signal{clk\_dis} activates (a condition that could lead to a glitch) there are two cases to consider. 
In the first one, \signal{na} goes fast to 0, which will delay the clock gating to the next cycle. 
In the second case, \signal{na} is slower than \signal{clk\_dis} and its propagation to node \signal{nc}, which will close \signal{TG2} and make \signal{na} return to 1. 
In this way, clock gating is free of hazards. 
The \signal{Clock Gating Circuit} allows keeping the output clock signal and enables DCO power reduction, by setting the \signal{comp} and \signal{fsel} signals to minimal values.

To avoid undesirable harmonic frequencies by possible bubbles in the ring, a reset process is necessary. 
Initially, the Controller block must generate the reset for the DCO and set both \signal{comp} and \signal{fsel} signals to their maximum values. 
This sets the feedback clock (\signal{fdbk\_clk}) signal and propagates this value faster to each ICO stage. 
Next, the DCO reset release occurs before removing reset signals from all other modules. 
The DCO architecture also allows the possibility of interrupting the clock without opening the ring, through the clock retain (CR) circuit, which is an exact copy of the DCO \signal{Clock Gating Circuit}. 
The \signal{clk\_retain} signal assertion does not stop the oscillation, instead, it disconnects the oscillation ring from the clock output. 
In this way, the CR circuit allows adjusting the clock to PVT variations effects, by disabling the clock signal propagation.
The external clock signal release happens as soon as the adjustment to PVT variations ends.

%#
%# SUB-SECTION 
%#==================

\subsection{The Actuator}
\label{subsec:actuator}

\begin{figure}[!t]
\centering
\includegraphics[width=0.75\linewidth]{figures/fsdac.pdf}
\caption{Structure of the DCO Frequency Selector DAC (FSDAC). Transistor \signal{M2} is always on, and \signal{fsel} controls the current mirrors.}
\label{fig:fsdac}
\end{figure}

The \signal{Actuator} converts the frequency selection digital input to an analog value that drives the ICO. 
The reason for using two DACs is functional: PCDAC enables limiting the maximum achievable operating frequency, while FSDAC manages the excursion between maximum and minimum frequencies, in a glitch-free manner. 
Since the ICO assumes a current-controlled structure, DACs manipulate current values as well.
The Frequency Selector DAC (FSDAC) produces the \signal{i\textsubscript{CTRL}} input to the ICO, and its internal structure appears in Fig.~\ref{fig:fsdac}.

The FSDAC receives as input a 15-bit thermometer-like encoded signal named \signal{fsel} that generates one out of 16 distinct current values in \signal{i\textsubscript{CTRL}}. 
Based on an input current source (\signal{i\textsubscript{COMP}}, from PCDAC) the number of active current mirrors that are connected in parallel defines the value of \signal{i\textsubscript{CTRL}}. 
All current mirrors are identical, i.e. supply the same amount of current and each bit of \signal{fsel} activates one of the current mirrors when set to 1. 
This code guarantees a monotonic conversion process, which provides continuously increasing or decreasing frequencies values. 
PVT variations can cause irregularities (\emph{e.g.} frequencies above the maximum allowed).

A gate-based switching scheme allows producing the required frequencies, and substitutes the classical drain-based switching of current mirrors (see Fig.~\ref{fig:fsdac}). 
This guarantees that the output current varies between minimum and nominal values, never exceeding the later. 
Thereby, it is possible to limit the maximum frequency generated to stay below or at a specified value. 
Transistor \signal{M2} is always on, to keep a minimum operating frequency. 
As input, the FSDAC has a reference current source (the structure to the left of transistor \signal{M2} in Fig.~\ref{fig:fsdac}) and the frequency selection channel, \signal{fsel}. 
The output is a controlled current (\signal{i\textsubscript{CTRL}}), which drives the ICO.

The PVT compensator DAC (PCDAC) enables fine tuning the reference current source (\signal{i\textsubscript{REF}}), generating \signal{i\textsubscript{COMP}}, which feeds the FSDAC.
An external controller can explore the direct manipulation of \signal{i\textsubscript{REF}} to provide extra precision, since its current adjusts the ICO delay. 
The PCDAC allows defining 256 distinct \signal{i\textsubscript{COMP}} levels to the FSDAC. 
In fact, this mechanism allows $\pm$10\% variations in \signal{i\textsubscript{REF}} without affecting the correct functionality of the DCO. 
The PCDAC has a circuit topology similar to that of the FSDAC (current mirrors connected in parallel). 
However, contrary to the later, it employs a binary code input (\signal{comp}) and a current mode output. 
A binary code is possible here because during DCO calibration, where the PCDAC has switching activity in its inputs, the generated clock signal is cut from the external entity. 
Thus, the LCG confines any possible hazard. 
Just as in the FSDAC, the PCDAC has a dedicated transistor to keep a minimum reference current.

\begin{figure}[!t]
\centering
\includegraphics[width=0.86\linewidth]{figures/dco-layout-labels.pdf}
\caption{DCO Layout. Visible layers are \emph{Poly}, \emph{Diffusions}, \emph{Metal1} and \emph{Metal2}.}
\vspace{-0.1cm}
\label{fig:layout}
\end{figure}

\begin{figure*}[!t]
\centering
\subfigure[]{
\includegraphics[width=0.31\linewidth]{figures/plot_a.png}
\label{fig:plot_a}}
\subfigure[]{
\includegraphics[width=0.31\linewidth]{figures/plot_b.png}
\label{fig:plot_b}}
\subfigure[]{
\includegraphics[width=0.31\linewidth]{figures/plot_c.png}
\label{fig:plot_c}}
\vspace{-0.6em}
\caption{a) frequency varying the PCDAC with the \signal{fsel} at maximum value, b) frequency at corners cases varying the \signal{fsel} and c) DCO power efficiency.}
\label{fig:charts}
\vspace{-0.3cm}
\end{figure*}

%#
%# SECTION 
%#==================

\section{Implementation and Quantitative Data}
\label{sec:impl}

% \begin{figure*}\centerline{
% \subfigure[Case I]{
% \includegraphics[width=2.5in]{subfigcase1}
% \label{fig_first_case}}
% \hfil
% \subfigure[Case II]{\includegraphics[width=2.5in]{subfigcase2}\label{fig_second_case}}}\caption{Simulation results}\label{fig_sim}\end{figure*


The DCO layout was developed using \emph{Cadence IC Design} and the STM CMOS 65~nm technology with low-power, standard and low $V_{th}$ transistors. 
The layout employs the three first metal layers, and \emph{Metal4} is used as a shield. 
As Fig.~\ref{fig:layout} shows, the layout of the DACs relies on the use of the common-centroid technique, widely used on current mirror and differential pair layouts to reduce mismatches \cite{Ra01}.
Guard rings were created around DACs, ICO and CR to avoid external noise from digital circuitry \cite{Ra01}.
The ICO has 75 transistors and the CR has 45 transistors, which amount to a total area of 850.6~\textmu m\textsuperscript{2}. This corresponds roughly to seventy 2-input NANDs of the technology core library. Clearly, the DCO is a small module, easily included as part of NoC routers and processing elements.

The DCO experiments were performed using \emph{Mentor Calibre PEX} to extract the parasitic capacitances. 
Three simulation scenarios were considered: \emph{worst}, \emph{nominal} and \emph{best} cases -- each one described in Table~\ref{tab:corners}. 
The \emph{best} and \emph{worst} corner temperatures were determined by experiments and are in accordance with \cite{Bh09} for deep submicron technology nodes. Simulations employed the \emph{Cadence Spectre}.
Fig.~\ref{fig:plot_a} illustrates the behavior of the PCDAC. 
Here, the \signal{comp} signal is varied along all values, while \signal{fsel} is set to its maximum (all 1s). 
It is possible from this to precisely define the maximum frequency that can be obtained for each value of \signal{comp}, which enables calibrating the DCO.
In all scenarios a 1~GHz frequency could be reached, even in the \emph{worst} corner. 
Fig.~\ref{fig:plot_b} shows the sixteen different frequencies provided by \signal{fsel} after setting \signal{comp}. 
The DCO power efficiency appears in Fig.~\ref{fig:plot_c}, and is between 618~nW/MHz and 120~nW/MHz.
Power consumption goes from 62~\textmu W, to 197~\textmu W, and drops to 0.45~\textmu W under clock gating. 


\input{tables/testcases.tex}
\input{tables/state-of-the-art.tex}



%#
%# SECTION 
%#==================
\section{Conclusion and Future Works}
\label{sec:conclusion}

%Figures highlighted in Sec.~\ref{sec:impl} corroborate the viability of using individual LCGs based on our DCO for modules such as routers and other individual IPs of an MPSoC, due to the LCG very small area and power overheads.
%The characteristics of the proposed DCO include support to clock gating, glitch-free frequency switching and PVT variation awareness. 
%The DCO needs an external current source, which implies additional area overhead. 
%However, it is possible to share this element among several LCGs. 
%Ongoing work includes the design of more complex controllers to take advantage of the 4096 frequencies which are in fact generated by the DCO (16 from the FSDAC $\times$ 256 from the PCDAC).

% GHeck: Texto simplificado que substituiria a conclusão e inclui uma análise da tabela 2

According to Table~\ref{tab:sota}, our DCO can operate with a reasonable number of frequencies while guaranteeing glitchless changes. 
Area occupation is higher than that reported in \cite{SLP08}, but it is at least 50\% lower than similar works built with DCOs \cite{APVP+11, HEH+13}.
Also  power consumption is at least 60\% lower than all other solutions and can still be improved by the clock gating mechanism.
In addition to the PVT adaptation features that Sec.~\ref{sec:impl} described, it is viable using individual LCGs based on the proposed DCO for modules such as routers and other IPs of an MPSoC.
The proposed DCO needs an external current source, which implies additional area overhead. 
However, it is possible to share this element among several LCGs. 
Ongoing work includes the design of more complex controllers to take advantage of the 4096 frequencies which are in fact generated by the DCO (16 from the FSDAC $\times$ 256 from the PCDAC).

% , and the development of software level control for LCGs in selected MPSoCs.


% \section{Acknowledgement}
% The authors would like to thank the ...

\balance
\bibliographystyle{IEEEtran}
\bibliography{bibliography}


% that's all folks
\end{document}
