Sociedade Brasileira de Dermatolodia Surgical & Cosmetic Dermatology

IR PARA

ISSN-e 1984-8773

Volume 5 Number 1


Back to summary

 

Original Article

Comparison of 2.5% thioglycolic acid, 2% hydroquinone, 2% Haloxyl, and 10% thioglycolic acid peeling in the treatment of periorbital hyperpigmentation

Comparação entre ácido tioglicólico 2.5%, hidroquinona 2%, haloxyl 2% e peeling de ácido glicólico 10% no tratamento da hiperpigmentação periorbital


Daniela Carvalho de Moraes Souza1, Cristiane Ludtke1, Emanuelle Rios de Moraes Souza1, Natana Werle Rocha1, Magda Blessmann Weber1, Ana Paula Dornelles Manzoni1, Fabiane Kamagai Lorenzini1

Dermatologist Physician; Intern at the
Dermatologic Surgery Program of the Faculdade
de Medicina do ABC (FMABC)—Santo André (SP),
Brazil1, Dermatology Resident at the Universidade
Federal de Ciências da Saúde de Porto Alegre
(UFCSPA)—Porto Alegre (RS), Brazil2, Intern at the Dermatology Service of the
Universidade Federal de Ciências da Saúde de
Porto Alegre (UFCSPA)3, Intern at the Dermatology Service of the
Universidade Federal de Ciências da Saúde de
Porto Alegre (UFCSPA)3, Associate professor of Dermatology and Head of
the Dermato-cosmiatryOutpatient Clinic of the
Universidade Federal de Ciências da Saúde de
Porto Alegre (UFCSPA)5, Visiting Instructorat the Dermato-cosmiatry
Outpatient Clinic of the Universidade Federal de
Ciências da Saúde de Porto Alegre (UFCSPA)6, Visiting Instructorat the Dermato-cosmiatry
Outpatient Clinic of the Universidade Federal de
Ciências da Saúde de Porto Alegre (UFCSPA)6

Received on: 3 July 2012
Approved on: 5 March 2013

The present study was carried out at the
Universidade Federal de Ciências da Saúde
de Porto Alegre (UFCSPA)—Porto Alegre
(RS), Brazil.

Financial support: None
Conflict of interest: None

 

Abstract

Introduction: Periorbital hyperpigmentation is a multifactorial dyschromia that involves the eyelid's vascularization and melanin hyperpigmentation. Objective:To compare the efficacy of 2.5% thioglycolic acid, 2% hydroquinone, 2% Haloxyl, and 10% thioglycolic acid peel in the reduction of dark circles. Methods: Eighty patients were divided into four groups. Groups 1, 2, and 3 underwent treatments with 2.5% thioglycolic acid, 2% hydroquinone, and 2% Haloxyl, respectively. Group 4 underwent five biweekly sessions of chemical exfoliation with 10% thioglycolic acid. All patients were evaluated by two physicians after the treatment: the one who performed the treatment, and another who was blinded to the outcome. The assessment was carried out using photographs and a clinical satisfaction questionnaire distributed to the patients. Results: The best results occurred in Groups 1 and 2. Group 3 had slightly inferior results, possibly due to the low concentration used. Group 4 had the least significant results, however this outcome may have been influenced by the higher age of the group members and the lack of daily use of medications. Conclusions: There is still not an ideal treatment for dark circles, with the currently available therapeutic resources only alleviating the difference in color between the lower eyelids and the rest of the face.

Keywords: HYPERPIGMENTATION, SKIN PIGMENTATION, EYELIDS

INTRODUCTION

Periorbital hyperpigmentation corresponds to the hyperchromia of the periocular region, being more often observed in women with dark hair. It is caused by genetically transmitted anatomical-physiological factors.1 There are two types of dark circles: vascular and melanic, nevertheless it is believed that most dark circles have mixed components, with hemosiderin and melanin being found in higher or lower quantities.2

The predominantly vascular dark circles have a pattern of dominant autosomal familial inheritance. It usually appears early in life, in childhood or adolescence. The diagnosis of this type of dark circles is carried out by pulling onthe lower eyelid so that there is a better visualization of the vessels under the skin, through transparency.2-4

The main structure involved in this type of dark circles is the excessive palpebral vascularization, which is derived from a number of vessels (supratrochlear, supraorbital, lacrimal, and dorsal nasal arteries, which in turn originate in the facial artery; angular artery originating in the facial artery; transverse facial artery, originating in the temporal superficial artery and its branches). Venous drainage follows an external pattern through veins associated with the various arteries, and an internal pattern, which enters the orbit through connections with the ophthalmic veins.5

In the vascular dark circle there is no alteration in the color of the skin; the eyelid is darker due to the visualization of dilated vessels through transparency.3 The cutaneous hyperchromia is believed to be a result of hemosiderin deposits, since it results from the biogenic transformation of a hemoglobin heme group when there is dermal blood extravasation, at which point the release of iron ions from that group occurs, resulting in the formation of free radicals, which in turn stimulate the melanocyte, generating associated melanic pigmentation.2

Smoking, alcohol consumption, mouth breathing, sleep deprivation, the use of vasodilator drugs, prostaglandin analogues- based eye drops, chemotherapy, and antipsychotics are factors that can contribute to this process through the stasis of blood vessels, leading to the alteration of the color in the region. The ideal treatment should include the suppression of triggering factors (when identified), the removal of pre-formed hemosiderin and the use of photoprotection.2,3

Predominantly melanic periorbital hyperpigmentation is that which occurs in older people with higher skin types. However, it can also occur in patients with lower phototypes— though they are generally more elderly—as a result of excessive and cumulative exposure to the sun.3

The main structure involved in this type of dark circles is the melanocyte—the cell responsible for the production of melanin. In dyschromias such as melasma and perhaps periorbital hyperpigmentation, melanocytes have special characteristics and are called "type-specific melanocytes" with larger, denser, and more elongated melanosomes. These are cells with "memory", i.e. when stimulated they increase their metabolism, helping to maintain the pigmentation process. The ideal treatment should also include the suppression of possible trigger drugs and cosmetics, and the removal of pre-formed melanin associated with the inhibition of melanogenesis, in addition to photoprotection.6

The importance of differentiating vascular dark circles from dark circles caused by melanic hyperpigmentation lies in the variation of therapeutic response. Dark circles caused by melanic hyperpigmentation are more sensitive to treatment, while the vascular type is more resistant and does not always present good results.3

Several depigmenting drugs have been prescribed for the topical treatment of dark circles, however there are few studies about the efficacy of such medicaments, comparative studies between them, or about the correlation between the results andthe patients'''''''' epidemiological characteristics.

Hydroquinone acts by increasing the excretion of melanocytes'''''''' melanin, reducing its production and inhibiting tyrosinase. There is still controversy about its indication for the treatment of periorbital hyperpigmentation, as in high concentrations it can cause pigmentation and opacification in the cornea.6-9

Few references have been made to the treatment of periorbital hyperpigmentation with hydroquinone. Among them, however, one combined study stands out. Itwas conducted with 18 patients using 5% hidroquinone and 0.1% retinoic acid for six weeks, followed by the application of Q Ruby laser, which resulted in a patient satisfaction level of 83.3% and evidenced a reduction in dermal pigmentation in all patients throughanatomical pathological examination.10

Thioglycolic acid (or mercaptoacetic acid) is indicated in the treatment of vascular dark circles at a concentration of 5% to 12%. It has an affinity to iron similar to that of the apoferritine, having the ability to chelate iron from hemosiderin, for presenting a thiol group.2

Haloxyl is an anti-dark circles active principle composed of matrikines, which stimulate the synthesis of components of the extracellular matrix (ECM), reinforcing the palpebral tonus, and of chrysin and N-hydroxysuccinimide, which act as chelators of bilirubin and iron respectively, decreasing local pigmentation. 11 In the present study, the authors employed a 2.5% thioglycolic acid stick, 2% hydroquinone cream, 2% haloxyl gel and serial peels of 10% thioglycolic acid gel as alternatives for the treatment of periorbital hyperpigmentation.

The present study was aimed at evaluating the clinical safety and efficacy of the above mentioned medications, and correlating results with the epidemiological factors and possible variants described in literature—such as dark circle type, presence of allergic diseases, alcoholism, smoking, and sleep deprivation— as aggravating factors of the periorbital hyperpigmentation.

METHODS

An open, non-paired, monocentric, non-randomized clinical trial was carried out with 80 volunteer patients aged 13-66, phototypes II to VI, of both genders, bearers of periorbital hyperpigmentation, who were randomly divided into four groups (of 20 patients each) in order to undergo the treatment. Sixty-two patients completed the treatment, where: G1 (2.5% thioglycolic acid, n = 18), G2 (2% hydroquinone cream, n = 14), G3 (2% Haloxyl gel, n = 15) and G4 (10% thioglycolic acid gel peel, n = 15).

The inclusion criterion was the desire to undergo treatment for periorbital hyperpigmentation, and the exclusion criterion was the topical application of the medications studied with a minimum frequency of no more than two times per week, non-attendance at all visits, and the presence of important side effects. The study was approved by the Research Ethics Committee of the Universidade Federal de Ciências da Saúde de Porto Alegre (RS), Brazil. All patients signed a term of informed consent about the study''''''''s phases and the possible treatment complications (erythema, edema, burning sensation, blistering, desquamation and microcrusting).

Patients in G1, G2, and G3 received topical treatment with 2.5% thioglycolic acid stick, 2% hydroquinone cram and 2% haloxyl gel, respectively, prescribed for daily home use with a minimum frequency of three times per week, for three months. The patients were examined fortnightly for the detection of side effects (erythema, edema, burning sensation, microcrusts) and the supply of medicaments.

For G4 patients, the authors adapted the protocol previously tested by Costa A. et al2. The volunteers underwent five sessions 10% thioglycolic acid peel, with an interval of two weeks. In the first session acid was applied for three minutes, with the exposure time being increased by 3 minutes at every new session, so that the duration of the last application session was 15 minutes.

The cleansing of the skin was carried out with 50% alcohol. A cotton swab was used for applying the substance to the lower eyelid. After the specified time, the substance was removed with gauze and then the excess was removed with water and then soap. Sunscreen was subsequently applied.

All patients received SPF 30 sunscreen, being instructed to apply it every four hours.

The patients were photographed under identical conditions of position, distance and lighting, before and after the treatment, and answered the questionnaire about epidemiological characteristics, side effects relative to the degree of satisfaction with the treatment with scores ranging from 0 (absence ofimprovement) to 10 (total improvement). The degree of clinical satisfaction was also evaluated in the last visit by the applicator physician and through photographs by a physician examiner, who was blinded to the outcome.

RESULTS
STATISTICAL TECHNIQUES EMPLOYED

The presentation of results was made through descriptive measures, such as absolute and relative distribution, as well as measures of central tendency and variability. The exploratory analysis of the distribution of data was performed using the Kolmogorov-Smirnov test (p>0.100).

The technique of analysis of variance (One-Way) was used for comparing the mean values among the four groups, with the basic assumption of homoscedasticity being verified by the Levene''''''''s test. In the situations in whichthere was not homogeneity of variances, the ANOVA was complemented by statistics derived through the Welch correction (used for comparing groups of different sizes), which is more robust regarding the violation of the assumption of homogeneity of the variance. The Bonferroni test, which is robust to heterogeneity of variances and groups of different sizes) was used for multiple comparisons. The data were statistically treated using the software SPSS 17.0 (Statistical Package for Social Sciences for Windows), with the adoption of a 5% significance level for decision criteria.

SAMPLE''''''''S PROFILE

The results presented were derived from a sample of 62 studied patients who were divided into four groups: G1 (2.5% thioglycolic acid stick, n = 18), G2 (2% hydroquinone cream, n = 14), G3 (2% Haloxyl gel, n = 15) and G4 (10% thioglycolic acid gel peeling, n = 15).

Evaluating the sample''''''''s profile regarding the groups, a statistically significant difference was detected in the comparison of the ages (p <0.001), suggesting that G4 (48.3 ± 13.6) had a significantly higher mean value than the other groups.

Regarding the exclusion criteria, 18 patients were excluded from the results. Only one patient (from G1) was excluded from the study because she had contact dermatitis after 14 days of the application of the medication. The other patients were removed from the sample for lack of assiduity in the use of the medication (for any reason, except for the presence of side effects) and failure to attend all medical visits to undergo clinical evaluation and to change medications.

ANALYSIS OF SATISFACTION AMONG GROUPS

Assessing the results regarding the satisfaction level, it was possible to detect the presence of a statistically significant difference (p <0.001) based on the analysis of the applicator physician''''''''s data, when comparing the groups. The analysis suggested that groups G3 (5.3 ± 1.6, p <0.01) and G1 (4.7 ± 1.9, p <0.05) had a satisfaction mean value significantly higher than that of G4 (2.6 ± 2.3). Thus, there was statistical evidence that the mean value for the satisfaction level was significantly higher in G1 and G3 when compared to G4. When comparing the patient''''''''s satisfaction level, the highest mean value of satisfaction was evidenced in G3 (7.0 ± 2.7), while the lowest mean value occurred in G4 (4.2 ± 3.1) (p> 0.05).

Regarding the analysis of data generated by the blinded evaluator, the average level of satisfaction proved higher in G1 (3.2 ± 2.4) than in the other groups, with statistical significance (p <0.05).

INTRAGROUP ANALYSIS—COMPARING INDEPENDENT EVALUATORS

When comparing the level of satisfaction among the evaluators (applicator physician, patient, and blinded evaluator) in each group, the results showed in general that the average levels of patient satisfaction were significantly higher than the averages estimated by the blinded evaluator (p <0.05). It was also found that the average satisfaction of the applicator physician and the blinded evaluator did not differ significantly, in each group, for the studied sample.

SATISFACTION LEVEL COMPARING THE TYPE OF DARK CIRCLES WITH THE TREATMENT EMPLOYED

A statistically significant difference was found in G1 when comparing the applicator physician''''''''s average level of satisfaction (p <0.05), indicating that the patients with the pigmented type of dark circles (6.8 ± 1.5) had an average satisfaction level higher than that of the patients with the vascular type of dark circles (3.6 ± 1.7) (p> 0.05).

According to the results observed in G2, G3, and G4, the average satisfaction was similar regardless of the type of dark circles.

When comparing the average level of satisfaction among groups for each type of dark circles, it was found that according to the applicator physician, the best results for the patients with mixed and pigmented dark circles occurred in G1 and G3 (p <0.05).

In the comparison among groups for the satisfaction reported by patients, there were no statistically significant differences (p> 0.05), indicating that the level of satisfaction was similar.

Regardingthe satisfaction presented by the blinded evaluator, there was a significant difference between the groups when comparing the mixed type of dark circles, with the better results taking place in G1 and G3. In the case of pigmented dark circles, the best results took place in groups G1 and G2.

SATISFACTION vs. SMOKING, ALCOHOL CONSUMPTION, SNORING/QUALITY OF SLEEP, AND ALLERGIES ACCORDING TOEACH GROUP

There was no statistically significant difference in the present study (p>0.05).

DISCUSSION

Hyperchromia of the orbital region, which lends a darkened qualityto the region, gives a tired look to the face. The result of the treatment of the skin with dark circles is most often gradual and rarely durable, since the physiopathology of this type of hyperpigmentation involves individual predisposition to pigmentation, in addition to the vasodilating physiological phenomena, which occur naturally and continuously.12,13

The palpebralcutaneous vasculature is composed of a complex system of branches originating in large vessels.5 Each time these small vessels are ruptured, a small volume of blood leaks into the dermis and settles in the hemosiderin deposit, which results from the biogenic transformation heme group of the hemoglobin. Simultaneously, the liberation of ferric ions and the stimulation of the formation of free radicals stimulate melanogenesis.2

In a general manner, the importance of differentiating vascular dark circles from melanic hyperpigmentation-based dark circles, resides in the expectation regarding the treatment outcome. Dark circles caused by melanic hyperpigmentation are more sensitive to therapy, since in this case the hyperchromia results from the focal epidermal melanocytic hyperactivity of clones of hyper-functioning melanocytes, with the consequent induced melanic hyperpigmentation—mainly by UV radiation. 14In this type of dark circles, the depigmentation obtained is reversible. In other words,the interruption of the treatment is enough for the melanin synthesis to become normalized. For that reason, it is important to make use of sunscreens during and after treatment.15 On the other hand, the vascular hyperchromias are more resistant due to the fact that they involve physiological phenomena, not always with good therapeutic results.3In the present study the best therapeutic results occurred with the mixed and pigmented dark circles, regardless of the treatment used, corroborating reports in the literature.

Currently there is a discussion about the real existence of vascular and melanic periorbital hyperpigmentation. It is believed that, to a greater or lesser degree, melanin and hemosiderin are present in nearly all types of dark circles.2,3

Although hydroquinone is the most effective drug in the treatment of melanodermias due to its inhibitory action of tyrosinasis, which prevents the conversion of tyrosine into melanin, 16 in the present study the 2.5% thioglycolic acid stick and the 2% Haloxyl gel had results superior to that of 2% hydroquinone in mixed dark circles or dark circles caused by melanic pigmentation (Figures 1, 2 and 3).

The low response to hydroquinone in patients with mixed and melanic hyperpigmentation can be attributed to the low concentration of the drug used. Generally, for products that are intended for facial application, the concentration normally used varies from 2% to 5%, however, it is known that 2% hydroquinone alone is poor in effectiveness, being more often indicated for maintenance treatment of hypercromia.6

Another important factor to be considered is the possibility of improper storage of the product. Hydroquinone is a highly effective substance for bleaching spots, however it has the inconvenience of chemical instability, being easily oxidated. 15 In the present study, patients received the medications fortnightly and were instructed about the necessary care to stabilize the product.

The 2.5% thioglycolic acid stick, the chemical exfoliation with 10% thioglycolic acid gel and the 2% Haloxyl gel are medications that act primarily in the vascular component of periorbital hyperpigmentation.

Thioglycolic acid is an alpha-hydroxy acid of great affinity with iron ion, which is chelated by the first. This therefore lends a potentially useful role to alpha-hydroxy acid in cases of hemosiderin deposits. The concentration used should not exceed 20%, however in the palpebral skin, lower concentrations (of up to 2.5%) are usually employed for daily use.16 It is worth noting that Costa et al. confirmed the efficacy and safety of the 10% thioglycolic acid gel peeling treatment, applied fortnightly in the palpebral region.2

The Haloxyl is composed by chrysin, N-hydroxysuccinimide (NHS) and matrikines–which are peptides released by the proteolysis of extracellular matrix''''''''s macromolecules. The matrikines stimulate the synthesis of extracellular matrix components (ECM), reinforcing the palpebral tonus, while the chrysin and the N-hydroxysuccinimide act as chelators of bilirubin and iron respectively, reducing local pigmentation.11

In the present study, the best results occurred in the pigmented and mixed dark circles, mainly in G1 (Figure 1) and G2 (Figure 2). It is believed that this result occurred because of the presence of hemosiderotic and associated melanic pigmentation in all types of dark circles. The improvement of the melanic pigmentation is also likely due to non-exposure to sunlight and the lack of regular use of sunscreen (prescribed for use every four hours, in all groups), during the treatment.

The most disappointing result occurred in G4 (Figure 4), showing that despite being a safe treatment option, it is perhaps not the best one, for at least in the present study, the periorbital hyperpigmentation was more responsive to daily topical treatment.

Several factors are described in the literature as aggravating the periorbital hyperpigmentation, including allergic diseases (atopic dermatitis, asthma, allergic rhinitis), sleep deprivation, and snoring, as well as habits like alcoholism and smoking.2,3 In the present study there was no statistically significant influence of those parameters on the results of the studied groups.

The groups were similar, except regarding the patients'''''''' average age, which was higher (48.3 ± 13.6) in G4 as compared to G1 (36.9 ± 13.4), G2 (33.6 ± 15.7) and G3 (26.7 ± 6.5). It is known that the dermis presents reduction of thickness with aging, with the loss of elastic fibers and collagen, which declines on average 2.1% per year. These alterationscan be responsible for the worsening of the unsightly appearance of dark circles and might have negatively influenced satisfaction in G4.17

Regarding the limitations of the study, the authors emphasize the absence of randomization, which implies the non-reduction of systematic errors (bias in the source of variation) and increases the influence of other factors (not controlled variables) that must have interfered in the effect detected in the study. As a result, there is not sufficient evidence underpinning the authors'''''''' hypothesis, explaining the results obtained.18

CONCLUSION

There is no definitive topical treatment for periorbital hyperchromia, because it is a dischromia of a multifactorial nature associated with a dynamic process that involves the palpebral vascularity and/or a personal tendency to melanic hyperpigmentation. Therapeutic resources currently available only atenuate the color difference between the lower eyelids and the remainderof the face.

In the present study, the best results took place in G1 and G2, especially in mixed and pigmented dark circles. G3 had slightly poorer results, perhaps due to the low hydroquinone concentration used. G4 obtained the lowest satisfaction, which may have been influenced by the older age of the patients (which adds an unaesthetic appearance to dark circles, caused by the presence of associated sagging in the eyelid) and also due to the absence of associated daily topical treatment.

It is important to note that given that this is an open study without placebo (control), thepresent study has limitations. The authors believe, however, that the investigation has fulfilled its primary objective, i.e.to signal the appropriateness of further pursuing this line of research.

References

1 . Sampaio SAP, Rivitti E. Dermatologia. 2 ed. São Paulo: Artes Médicas, 2000. p. 277

2 . Costa A, Basile DVA, Medeiros VLS, Moisés AT, Ota SF, Palandi JAC. Peeling de gel de ácido tioglicólico 10% opção segura e eficiente na pigmentação infraorbicular. Surg Cosmet Dermatol. 2010;2(1): 29-33

3 . Denisesteiner.com.br [Internet]. São Paulo: Clínica Denise Steiner; c2004 [acesso 2013 Fev 28]. Disponível em: http://www. denisesteiner.com.br/derma_estetica/olheiras2.htm .

4 . Freitas FM, Cestari TF. What causes dark circles under the eyes? J Cosmet Dermatol. 2007; 6(3):211-5

5 . Richard LD, Wayne V, Adam WMM. Grays - Anatomia para estudantes 2005. p.831

6 . Kede MPV, Sabatovich O. Dermatologia Estética São Paulo: Atheneu; 2004. p. 259

7 . Medicinanet.com.br. [Internet]. São Paulo: Medicina Net. [acesso 2013 Fev 28]. Disponível em http:// www. medicinanet. com.br/bula/5172/tri_luma.htm.

8 . Germedpharma.com.br [Internet]. São Paulo: Germed. [acesso 2013 Fev 28]. Disponível em: http:// www. germedpharma. com.br/site/uploads/tx_productspharma/082690_Hormoskin.pdf

9 . Medley [Internet]. São Paulo: Medley. [Acesso 2011 Abril 12]. Disponível em: http://www.medley.com.br/portal/bula/triderm_creme_15g.pdf .

10 . Momosawa A, Kurita M, Ozaki M, Miyamoto S, Kobayashi Y, Ban I, Harii K. Combined Therapy Using Q-Switched Ruby Laser and Bleaching Treatment with Tretinoin and Hydroquinone for Periorbital Skin Hyperpigmentation in Asians. Plast Reconstr Surg. 2008;121(1):282-8

11 . Mapric.com.br [Internet]. São Paulo: Mapric. [Acesso 2013 Fev 28]. Disponível em: http://www.mapric.com.br/anexos/boletim465_ 14112007_081118.pdf. .

12 . Teixeira V. Treatment of idiopathic Cutaneous Hypercromia of the orbital region (ichor) with erbium laser: a retrospective assessment. Arq Catarinenses Med. 2007;36(Suppl 1):76-9

13 . Nicoletti MA, Orsine EMA, Duarte ACN, Buono GA. Hipercromias: aspectos gerais e uso de despigmentantes cutâneos. Cosmetics & Toiletries. 2002; 14:46-51. [Acesso 2013 Fev 28]. Disponivel em: http://www.tecnopress- editora.com.br/pdf/nct_443.pdf .

14 . Miot LDB, Miot HÁ, Silva MG, Marques MEA. Fisiopatologia do melasma. An Bras Dermatol. 2009; 84(6): 623-35

15 . Kato FP. Verificação do prazo de validade de cremes contendo hidroquinona preparados magistralmente: evidências do processo de oxidação. Rev Ciênc Farm Básica Apl. 2010;31(2):199-203

16 . Notaroberto P. Manejo das hipercromias de maior interesse em angiologia. Revista de Angiologia e Cirurgia Vascular. [Acesso 2013 Fev 28]. Disponível em http: //www.sbacvrj.com.br/paginas/revistas/pdf/2006/5/interface.pdf

17 . Medicina geriátrica, geriatria e gerontologia [Internet]. [Acesso 2013 Fev 28]. Disponivel em : http:// www. medicinageriatrica. com.br/ 2007/05/05/envelhecimento-cutaneo

18 . Campbell DT, Stanley JC. Experimental and quasi-experimental designs for research. Chicago: Rand McNally; 1966


Facebook Twitter Linkedin

© 2021 Sociedade Brasileira de Dermatologia - Todos os direitos reservados

GN1 - Sistemas e Publicações