Sleep disorder is a medical disorder of the sleep patterns of a person and involves problems including snoring, sleep apnea, insomnia, and sleep deprivation, which can affect an individual’s health, safety and quality of life [1]. Among these types of sleep disorders, insomnia is the most common disorder characterized by symptoms such as difficulty in falling asleep and/or staying asleep, and abnormalities in quality and quantity of sleep, resulting in serious impairments in daytime activities, such as fatigue, difficulties with cognitive functions, and mood disturbances during wakefulness [1,2]. Insomnia has been categorized as primary (idiopathic) and secondary (comorbid) insomnia [3], depending on whether the condition of insomnia is attributable to other medical or psychiatric disorders by the Diagnostic and Statistical Manual for Mental Disorders 4th edition (DSM-IV) [4] and International Classification of Sleep Disorders: Diagnostic and Coding Manual 2nd edition (ICSD-2) [5]. The DSM-V [6], however, proposed on the importance of comorbid nature of insomnia and calls for treatment of both insomnia and the medical disorder [7]. The US National Comorbidity Survey showed that there is a high rate of comorbidity between insomnia and medical and psychiatric disorders such as anxiety, depression, and substance misuse disorders [8]. Also, in longitudinal studies about insomnia [9], they suggested that it could be a risk factor for new onset psychiatric disorders. Breslau et al. [10] and Roth et al. [9] reported that individuals with insomnia were four times more likely to develop new major depression over the next 3–5 years than were individuals without insomnia.

The treatment of insomnia includes cognitive behavioral therapy (CBT), medication, or a combination of both [3]. In the case of acute insomnia, pharmacological treatment is mainly performed, and in the case of chronic insomnia, CBT such as sleep restriction, stimulus control, relaxation, and cognitive strategy is recommended [2,10]. However, according to 2005 US National Institutes of Health state of the science conference on insomnia, both the treatment modalities of CBT and pharmacotherapy are more effective in managing chronic insomnia than monotherapy of CBT [11]. Hypnotic agents approved by the US Food and Drug Administration (FDA) for the treatment of insomnia include bezodiazepine receptor agonists (zolpidem, estazolam, and triazolam), nonbenzodiazepine modulators of gamma aminobutyric acid-A (GABA-A) receptors (esxopiclone and zaleplon), antihistamine drugs (hydroxyzine and diphenhydramine), a tricyclic drug (doxepin), and a melatonin receptor agonist (ramelteon) [1,12] (Table 1). Another drugs to improve symptoms of insomnia approved by the FDA are barbiturates (secobarbital and butalbital), but they are not recommended because of potential toxicity [1,12,13]. In addition, there are a number of drugs without FDA indication for insomnia, including sedating antidepressants (e.g., trazodone, mirtazpine, and nefazodone) that are sometimes used to treat comorbid depression patients with insomnia [13]. Several serotonin 5-HT₂ receptor antagonists also provide favorable effects on depression and insomnia by increasing slow wave sleep (SWS) and decreasing waking after sleep onset [7,14]. Serotonin (5-HT) within central nervous system contributes to numerous physiological functions including sleep-wake behavior, cognition, circadian rhythm, pain, and depression. 5-HT receptor can be classified into fifteen 5-HT receptor subtypes, which mediate all of these physiological processes [15-17]. The role of 5-HT in sleep regulation has been extensively studied and many studies proposed that 5-HT participates in promotion of wakefulness (W) and regulation of rapid eye movement (REM) sleep [16]. Trazodone (Figure 1) belongs to 5-HT_2A receptor antagonists and acts as an α1 adrenergic receptor antagonist and as an inhibitor of serotonin transporter (SERT), indicating efficacy for insomnia [18,19]. One of 5-HT_2A receptor antagonists, lumateperone (ITI-007) (Figure 1) behaves as a dopamine receptor modulator and a serotonin transporter inhibitor and improves quality of sleep [20,21]. Hagan et al. [22] demonstrated that a 5-HT₇ receptor antagonist, SB-269970 (Figure 2) increased the latency to REM sleep and decreased the total duration in REM sleep. There is a further investigation that 5-HT7 receptor knock-out mice spent less time in REM sleep than wild-type mice [23]. LP-44 [24] (Figure 2), a 5-HT₇ receptor agonist, was studied for effects on sleep in rats. Administration of LP44 into dorsal raphe nucleus (DRN) induced decrease in REM sleep and the number of REM periods [25]. Bonaventure et al. [26] reported that the preclinical and clinical research of 5-HT₇ receptor antagonist, JNJ-18038683 (Figure 2) leads to dose-dependently suppress REM sleep and increase REM sleep latency after the treatment. Kim et al. [27] have discovered selective β-arrestin biased ligand against 5-HT₇ receptor (Figure 2), which increased non-REM (NREM) sleep duration unlike a balanced antagonist SB-269970. In this review, we focus on the role of serotonin (5-HT) receptor in insomnia and effects of the related drugs on sleep.

One of serotonin receptors, 5-HT₂ receptor is comprised of 5-HT_2A, 5-HT_2B, and 5-HT_2C receptor subtypes [28,29]. According to autoradiographical and immunohistochemical studies, these 5-HT₂ receptors are distributed in specific area of the brain. 5-HT_2A receptors are located in basal fore brain, ventral tegmental area, and median raphe nuclei linked to the promotion of the waking state [29,30]. Several studies have shown that 5-HT₂ receptor is responsible for the regulation of SWS in human and animals [24, 28]. 5-HT_2A receptor knock-out mice showed an increase of W and a decrease of NREM sleep. Values of REM sleep were not altered by genotype [31]. Dugovic and Waquier [32] reported that multiple 5-HT₂ receptor antagonists induced increase of SWS in rodents, but drugs with 5-HT₂ receptor agonism decreased SWS [29,30]. A 5-HT_2A receptor antagonist, trazodone (Figure 1) might be a useful drug to reduce perceived sleep disturbances because people who is suffering from insomnia have reduced pattern of SWS [33-35]. Although the FDA confirmed that trazodone was effective against depression not insomnia, it was recently prescribed as a sleep aid [34]. Trazodone has potent binding property of 5-HT_2A receptor with antagonism and acts as an inhibitor of SERT, as well as antagonist of α1 adrenergic receptor and H1 histamine receptor. When 5-HT_2A receptor is over-saturated by the range of 25–100 mg doses of trazodone, α1 adrenergic receptor and H1 histamine receptor are simultaneously blocked, recruiting the hypnotic actions which promote sleep onset and maintain sleep [20]. Inhibition of these neurotransmitter systems involved in the arousal mechanism can impair arousal and induce sleep maintenance without causing tolerance [20]. There are a number of reports that clinical trials of trazodone are conducted to evaluate effectiveness in the treatment of insomnia. In order to examine the hypnotic efficacy and daytime effects of trazodone, Nierenberg et al. [36] and Roth et al. [37] conducted clinical trials on patients with primary insomnia. After administration of trazodone for a week, the results showed that total awakenings were significantly decreased and SWS time was increased, without change in sleep latency and REM latency, and REM sleep [36,38]. Consequently, it has been concluded that trazodone had an improved effect on insomnia compared with placebo [37]. Muratatorio et al. [38] performed a polysomnographic study of patients with depression and insomnia treated with trazodone. In this study, it seems that trazodone is effective in alleviating depression symptoms and improving insomnia. Roth et al. [37] used overnight polysomnography and modified Multiple Sleep Latency Tests to assess the quality of sleep for trazodone. Their polysomnographic data showed that night-time awakenings and perceived sleep difficulty are improved after low dose (50 mg) trazodone, and trazodone also increases SWS in primary insomnia [37]. Thus, these results are consistent with previous findings of trazodone’s effects on sleep quality [39,40]. Since sleep disturbances seen in depression are a side effect of antidepressants, a number of depressive patients who are taking selective serotonin reuptake inhibitors (SSRIs) are suffering from insomnia. Therefore, Kaynak and his colleuges [41] assessed the effects of trazodone on depressive patients with insomnia treated with SSRIs. They administered 100 mg trazodone or placebo for a week to subjects who had been treated with SSRIs for at least 3 weeks and measured the quality of sleep with using polysomnographic recordings after treatment with drugs. Trazodone produced significant improvement in sleep parameters with increase in total sleep time (TST), sleep efficiency index, sleep continuity index, and SWS and reduction of the number of awakenings. Combined treatment with CBT and various medication for primary insomnia has not been shown to produce greater satisfaction with sleep complaints than single treatment of CBT [42]. Zavesicka et al. [42] have chosen trazodone to combine with CBT based on such previous research that it has feasibility for long-term use and positive effects on sleep architecture. Combined treatment with trazodone significantly increased SWS duration and TST in 20 patients with chronic primary insomnia, while this influence is not observed in the group of patients treated with CBT only [43]. Another potent 5-HT_2A receptor antagonist, lumateperone (ITI-007) (Figure 1) is a new and atypical antipsychotic with strong affinity for dopamine D₂ receptor and SERT. Lumateperone is characterized by the binding affinities to 5-HT_2A receptor (K_i 0.54 nM) and D2 receptor (K_i 32 nM) or SERT (K_i 61 nM), while binding affinities are relatively low at histaminergic H1, serotonin 5-HT_2C, and muscarinic receptors associated with side effects of antipsychotic drugs [43,44]. In addition, lumateperone induced SWS at low doses (<0.2 mg/kg in rodents), and was considered to relieve insomnia without daytime sedation. As the dose is increased (>1.0 mg/kg in rodents), other drug targets such as dopamine D₂ receptor and SERT are saturated by lumateperone with relieving symptoms related to schizophrenia or bipolar disorder. These data indicated that lower concentration of lumateperone might have predominant 5-HT_2A receptor selectivity with moderate interaction with D₂ receptor and SERT and that as doses of the drug is increased, additional pharmacological effect also emerged [43]. Though lumateperone interacts with 5-HT_2A, D₂ receptor and SERT in a dose dependent manner, it presents a unique pharmacology and influences neuropsychiatric and neurodegenerative diseases including sleep disturbances in neuropsychiatric diseases without off-target interaction and other side effects [45]. Davis et al. [46] proposed metabolic pathway in which lumateperone is mainly reduced to its secondary alcohol derivative (IC200131) by a ketone reductase and is converted to its N-desmethyl derivatives (IC200161) or its reduced derivative (IC200565) by cytochrome P450 3A4 (CYP3A4) and IC200131 can be converted back to lumateperone via CYP3A4. Pharmacological efficacy of the drug is driven by these active metabolites. The phase I, phase II, and phase III clinical trials of lumateperone have been completed, thus Intra-Cellular Therapies Inc., the developer of lumateperone, recently submitted the New Drug Application (NDA) to the U.S. FDA. In phase I studies, oral doses of lumateperone were administered to healthy volunteers and patients with schizophrenia, which were safe and had little adverse effects [45]. Based on brain receptor occupancy studies using positron emission tomography, 60–120 mg doses were selected for further investigation and in Phase II B study, 60 mg dose of lumateperone considerably reduced insomnia, dementia, and symptoms related to schizophrenia. In Phase III clinical trials, lumateperone showed superior antipsychotic efficacy for treatment of schizophrenia compared with placebo and indicated improved social functioning measured in the Personal and Social Performance Scale compared to placebo [47]. Lumateperone currently received Fast Track designation from the FDA in November 2017 for the treatment of schizophrenia. Intra-Cellular Therapies Inc. announced that the company has completed the rolling submission of NDA to the U.S. FDA for lumateperone. The NDA submission is supported by data from 20 clinical trials and more than 1,900 subjects exposed to lumateperone [48].

5-HT₇ receptor is the last 5-HT receptor subtype discovered and has been cloned from various species such as human, rat, rabbit, and guinea pig [49-52]. 5-HT₇ receptor is positively coupled to adenylyl cyclase via activation of G_S, inducing an intracellular increase of cAMP [49,50]. According to immunohistochemistry and autoradiography, 5-HT₇ receptor are distributed in hypothalamus, pontine nuclei, DRN, and hippocampus involved in the regulation of the sleep-wake cycle including SWS, REM sleep, and W [49-52]. 5-HT₇ receptor in the DRN are localized to GABAergic neurons not to serotonergic neurons, suggesting that it does not subserve an autoreceptor fuction [53]. In the 5-HT₇ receptor knock-out mice, 5-HT₇^-/- mice, total time and episode frequency of REM sleep are reduced than wild type, 5-HT₇^+/+ mice with no difference between genotypes in parameters of W or SWS [23]. Microinjection of a 5-HT₇ receptor antagonist, SB-269970 (Figure 2) into the DRN increased W and reduced REM sleep and the number of REM periods compared with those of the control vehicle. Furthermore, systemic administration of SB-269970 reduced the time spent in REM sleep and the number of REM sleep occurrences and increased the latency to REM sleep [54,55]. Excitatory 5-HT₇ receptors expressed by GABAergic cells is suppressed by microinjection of SB-269970 into the DRN, which leads to the reduction of GABAergic inhibition of 5-HT neuron and to the increase of 5-HT release at postsynaptic sites critical for the induction and maintenance of REM sleep, resulting in decrease of REM sleep [55]. This proposal is supported by previous studies that pretreatment with the GABA-A receptor agonist muscimol prevented the effect of SB-269970 on REM sleep [56]. Monti et al. [25] demonstrated that the microinjection of LP-44 (Figure 2), a 5-HT₇ receptor agonist into the DRN induced the reduction of REM sleep and the number of REM periods. Similar results were observed in microinjection of SB-269970 and pretreatment with SB-269970 inhibited the effects of LP-44 on REM sleep [25]. Thus, they tentatively proposed that since numerous GABAergic neurons in the DRN contribute to long projections reaching the laterdorsal and pedunculopontine tegmental nuclei (LDT/PPT) containing cholinergic neurons, the activation by LP-44 of projection GABAergic neurons could cause inhibition of cholinergic neurons in the LDT/PPT, resulting in reduction of REM sleep [25]. Administration of JNJ-18038683, a 5-HT₇ receptor antagonist into rats dose-dependently suppressed REM sleep and the duration of REM sleep and significantly increased REM sleep latency after the treatment without changes in NREM sleep [26]. This group also found that JNJ-18038683 enhanced REM suppression induced by citalopram both in rodents and in humans [26]. In addition, clinical evaluation has shown that JNJ-18038683 significantly increased the time of REM sleep episode compared to placebo and reduced total REM sleep time. These data demonstrated that the effect of 5-HT₇ receptor blockade induced by JNJ-18038683 on REM sleep is translated from rodents to humans [26,57]. JNJ-18038683 is now undergoing a phase II clinical trials as adjunctive treatment to improve cognition impairment and to alleviate depressive symptoms in stable bipolar patients [57]. Kim et al. [27] have discovered Azepine (3-(4-chlorophenyl)-1,4,5,6,7,8-hexahydropyrazolo[3,4-d]azepine) as the β-arrestin biased ligand against 5-HT₇ receptor [Ki 30 nM, IC₅₀ (G protein) 7,800 nM, EC₅₀ (β-arrestin) 162 nM] through the functional selectivity tests. They also carried out electroencephalogram experiment to investigate a correlation between β-arrestin biased agonism of Azepine and NREM/REM sleep. Azepine produced increase of NREM sleep duration and reduction of REM sleep duration, while the balanced antagonist SB-269970 decreased REM sleep duration without change of NREM sleep. Although they need more experimental data to identify the correlation between biased ligand and sleep, they suggested that β-arrestin activity of 5-HT₇ receptor might modulate NREM sleep patterns based on those results.

In this review, we have explored and investigated the role of 5-HT receptors and effects of related drugs or drug candidates on insomnia. We studied among those 5-HT receptors, especially, 5-HT_2A receptor and 5-HT₇ receptor with the possibility of treating insomnia. 5-HT_2A receptor antagonist, trazodone promoted sleep onset and is effective for sleep maintenance by increasing SWS and decreasing night-time awakenings with no significant changes in other sleep parameters. Lumateperone, recently developed as 5-HT_2A receptor antagonist, indicated improvement in sleep quaility in phase II B clinical trials, received Fast Track designation from the FDA in November 2017, and currently submitted NDA to the U.S. FDA. This review article focused on administration of agonists or antagonists against 5-HT₇ receptor, representatively SB-269970, JNJ-18038683, and LP-44 into DRN which altered sleep patterns in rats. These ligands reduced REM sleep and increased REM sleep latency without changes in W or SWS. Furthermore, Azepine has been discovered as β-arrestin biased agonist, which induced increase of NREM sleep duration and reduction of REMS duration. Taken together, 5-HT receptors has been involved in sleep and various drugs or drug candidates binding to 5-HT receptors could change sleep parameters, thus these findings from previous or current studies suggest that 5-HT receptors are promising of the treatment of insomnia and that there is an urgent need for further research about correlation between 5-HT receptors and insomnia.