Abstract
Free-space optics (FSO) is being considered for 5G and next-generation wireless networks for its free spectrum, short response time, high data rates, and innate security. We designed a dual-polarization FSO system with adaptive modulation at the receiver and advanced digital signal processing (DSP) to mitigate atmospheric turbulence and polarization impairments. Simulation results demonstrate that adaptive modulation significantly improves bit error rate (BER) and spectral efficiency under varying turbulence levels. A full DSP algorithm, including filtering, equalization, and carrier recovery, reduces BER by orders of magnitude in severe conditions. We analyze system performance under various weather scenarios (clear air, haze, fog) and polarization-related impairments (polarization-dependent loss, crosstalk, and rotation). The proposed dual-polarization FSO system achieves high data rates (≥70 Gbps) over extended distances (∼10 km) while maintaining acceptable BER. These results provide valuable insights for integrating FSO links into 5G and future networks, highlighting the benefits of polarization multiplexing, adaptive techniques, and DSP in overcoming FSO channel challenges. The findings will aid network planners in deploying robust high-capacity FSO links for next-generation cellular networks.
Introduction
The explosive growth of 5G and future wireless networks demands ultra-high capacity, low-latency back-haul and front-haul links. Free-space optics (FSO) communication has emerged as an attractive solution for these links because it offers gigabit-per-second data rates, unlicensed spectrum usage, minimal electromagnetic interference, and strong security due to the narrow optical beam (Sharma et al., 2025). Unlike microwave links, FSO links give you fiber-like transmission speeds wireless and are very useful when running fiber is difficult. In particular, for 5G back haul and disaster recovery, FSO has become a key facilitator in urban settings (Kulshreshtha & Garg, 2020). Yet, the way FSO systems function greatly depends on weather conditions. Atmospheric turbulence and weather phenomena (fog, rain, haze) induce random intensity fluctuations and attenuation in the optical beam, leading to increased BER and reduced link reliability (Prokeš, 2009). These channel impairments, caused by variations in the refractive index along the path (optical scintillation) and absorption/scattering by particulates, pose the primary challenge for FSO communications. Field trials have reported 100 Gbps coherent FSO links in the atmosphere and ∼1 Tbit/s over 53.4 km terrestrial free space with coherent modulation and full adaptive optics; thus our ≥70 Gbps (simulation) target lies within demonstrated FSO capacity envelopes. FSO also operates in an interference-free optical spectrum, which supports such high throughputs (Elamassie & Uysal, 2023; Horst et al., 2023).
At the beginning, researchers modeling FSO channels treated loss due to turbulence with log-normal and Gamma-Gamma statistics. Such models predict that under strong turbulence, the optical signal experiences deep fades that can severely degrade an FSO link's throughput and availability (Esmail, 2021). Furthermore, where coherent systems include phase recovery and error correction, the receivers lack these features and are therefore more sensitive to channel disturbances (Karar et al., 2023). In addition to atmospheric turbulence, dual-polarized FSO systems face polarization-specific impairments Mismatches in polarization angle alignment, polarization-dependent loss (PDL) in optical components, and polarization crosstalk between channels can all degrade the separation of the two polarization-multiplexed data streams, these effects make signal recovery at the receiver more difficult, as the two polarized channels may not remain perfectly orthogonal after propagating through the atmosphere and optical hardware (Brima et al., 2021).
Researchers have proposed various mitigation techniques to meet these challenges, Adaptive optics and beam shaping can compensate for wavefront distortions, while spatial diversity (multiple apertures or multi-beam systems) can average out fading Channel coding and interleaving at the protocol level improve FSO link reliability under fades (Selim et al., 2026). Importantly, adaptive modulation has been explored to maintain link performance under time-varying channel conditions. In adaptive modulation, the transmitter or receiver dynamically adjusts the modulation format based on the current channel quality (e.g., switching to a lower-order modulation during deep fades to maintain a low BER). This can significantly improve the average throughput of FSO links under turbulence by exploiting periods of good channel conditions with high-order modulation and using robust modulation during bad periods. Recent studies have shown that adaptive modulation combined with coding can approach the capacity of FSO channels across a range of turbulence conditions (Sahrab & Albasri, 2026).
Another powerful approach is to leverage digital signal processing (DSP) techniques at the receiver, analogous to those used in coherent fiber-optic systems. By filtering and equalizing the received optical signal, one can correct inter symbol interference, residual frequency offsets, and mixing of polarization channels. For example, decision feedback equalizers or LMS adaptive filters can mitigate the effects of multi path and scintillation-induced distortions in IM/DD FSO links (Sano et al., 2025). In dual-polarization systems, MIMO DSP algorithms used in polarization-division multiplexed fiber links (e.g., polarization de multiplexing and orthogonality via constant modulus or CMA algorithms) can be applied to FSO to separate the polarization channels despite crosstalk. Carrier phase recovery and clock recovery algorithms, common in coherent optical receivers, can further improve performance if coherent detection is employed. Prior work on dual-polarization FSO has utilized DSP to implement self-coherent reception, demonstrating improved sensitivity and robustness compared to direct detection (Vyas & Kumar, 2025).
Numerous researchers have investigated the performance of FSO systems under different conditions. Early studies such as Henniger and Wilfert provided a broad introduction to FSO architectures (Henniger & Wilfert, 2010), While Khalighi and Uysal surveyed channel modeling approaches, pointing errors, and modulation techniques (Khalighi & Uysal, 2014). More recent contributions have emphasized MIMO and diversity-based solutions. For instance, Kaushal and Kaddoum reviewed spatial diversity and coding strategies to mitigate turbulence and Kaushal and Kaddoum (2016). Huang et al. demonstrated multimode aperture diversity to combat turbulence (Huang et al., 2021). In Parallel, Ghassemlooy et al. presented comprehensive channel and system modeling frameworks, offering MATLAB-based evaluations that remain widely used (Ghassemlooy et al., 2019). On the modulation side, Djordjevic analyzed adaptive modulation and coding schemes to improve FSO link resilience (Djordjevic, 2010). Recent work by Kumar et al. focused on dual-polarized 16-QAM systems under various weather scenarios, highlighting the importance of polarization-aware designs (Kumar et al., 2024). Similarly, Alouini and Ata analyzed high-altitude platform station (HAPS)-based FSO links with adaptive optics, stressing their role in future beyond-5G architectures (Ata & Alouini, 2022).
Studies have also begun integrating digital signal processing concepts from coherent fiber optics into FSO. Kedar and Arnon and Bloom et al. discussed early challenges in urban FSO deployments, whereas contemporary efforts employ DSP-based equalization and phase recovery to counteract turbulence and polarization mixing (Bloom et al., 2003; Kedar & Arnon, 2004). This body of literature underscores the progression from basic FSO feasibility demonstrations toward advanced adaptive, polarization-multiplexed systems suitable for 5G/6G backhaul. Recent works have pushed beyond point-to-point terrestrial FSO into relay/hybrid architectures and space–air–ground integrated networks (SAGIN). Xu et al. analyze a cooperative RF/FSO SAGIN with adaptive combining at relays, deriving performance for multi-segment heterogeneous links and showing reliability gains from joint RF/FSO selection and combining. Their focus is network-level cooperation rather than polarization multiplexing or receiver-side DSP on a single optical hop (Xu et al., 2024).
The recent studies have covered hybrid RF-FSO links with re configurable intelligent surfaces (RIS) to improve the reliability of links and spectrum efficiency in changing atmospheric conditions (Vishwakarma et al., 2024). They can be used to solve the co-channel interference (CCI) and non-line-of-sight (NLOS) problems through the dynamic adaptation of propagation paths. In addition, chaotic-based modulation, such as differential chaos shift keying (DCSK), and other chaos-based modulation schemes have been used in FSO systems to enhance signal security and stability to atmospheric turbulence (Verma et al., 2024). These works offer useful information on how to come up with strong hybrid architectures, and their optimization models are not the same as the current FSO only transmission architecture.
This paper presents a comprehensive investigation of a DP-FSO communication system employing advanced DSP technique and adaptive modulation. The key contributions of this work include: Development of a comprehensive DP-FSO system model incorporating atmospheric turbulence effects, polarization impairments, and realistic system parameters Implementation and evaluation of DSP algorithms to remove channel impairments Development of an adaptive modulation scheme that dynamically adjust modulation techniques Validation and optimization of system at different channel conditions
The remainder of this paper is organized as follows. Section 2 presents the methodology of the proposed system. Section 3 describes system model. Section 4 illustrates results and discussions and Section 5 concludes the paper.
This work is the first, within our scope, to treat polarization multiplexing, adaptive modulation, and receiver-side DSP together in a single long-reach FSO model targeted at 5G back/front haul. We explicitly incorporate Gamma–Gamma turbulence, fog/rain attenuation, pointing error, and polarization impairments (rotation, PDL, crosstalk), then quantify their combined impact on BER, EVM, capacity, and achievable data rate across QPSK–256-QAM. The analysis yields practical operating thresholds and distance–rate trade-offs for ≈10 km links, including meeting a 70-Gbps requirement in clear air and identifying when dual-polarized 256-QAM can reach 160 Gbps. These results turn the system description into deploy able guidance rather than isolated component studies.
Methodology of the Proposed System
We consider a dual-polarization FSO communication system in which two data channels are transmitted simultaneously on orthogonal polarization states of a laser beam. At the transmitter, a high-power laser diode (or modulated optical source) is split into two orthogonally polarized beams (e.g., horizontal and vertical polarization). Each beam is independently modulated with an intensity-modulation format (such as on–off keying or M-ary QAM) carrying a separate digital data stream. The two polarized optical signals are then combined and launched into the free-space channel through a telescope. This effectively doubles the aggregate data rate by using polarization multiplexing, like dual-polarization transmission in fiber optics (Sahu & Sahu, 2026).
In our design, the transmitter supports multiple modulation formats up to 256-QAM on each polarization, enabling very high spectral efficiency when channel conditions permit. A pointing and tracking subsystem keeps the transmitter laser aligned with the receiver aperture, as FSO links require line-of-sight alignment and can be susceptible to beam wander due to building sway or atmospheric effects. At the receiver, a telescope lens collects the incoming dual-polarized beam and focuses it on a polarization-multiplexer module. This could be a polarization beam splitter that separates the horizontal and vertical polarization components of the optical field. Each polarization is then detected by a photo diode (for direct detection) or sent to a coherent receiver (if using coherent detection). In our implementation, we assume intensity modulation with coherent detection for simplicity – a common choice for FSO due to its relative simplicity and cost-efficiency. The photo-diode outputs two electrical signals corresponding to the two polarization intensities. Ideally, with perfect polarization alignment, each photo diode sees only its intended signal. In practice, misalignment can cause polarization crosstalk where each photo diode gets a mixture of both signals. We model polarization crosstalk by coupling factors (in dB) representing the leakage between channels (Cao et al., 2022). We also include a polarization-dependent loss factor to represent any asymmetric loss between polarization channels (e.g., due to optics or unequal photo diode responsivity).The transmitter/receiver apertures, beam divergence, and power levels adopt typical FSO design ranges summarized by Henniger & Wilfert and Bloom et al.; they are not tied to a particular product but match widely cited link-budget practice (Bloom et al., 2003; Henniger & Wilfert, 2010). Figure 1 depicts the basic block diagram of proposed system (Mouhammad et al., 2026).

Block diagram of proposed system.
In this section, we present the system model used in the FSO system and exhibit the principle of our proposed dual-polarization FSO system for 5G and beyond, incorporating adaptive modulation and advanced Digital signal processing (DSP).
The atmospheric turbulence is modeled using the Gamma-Gamma distribution, which considers both small-scale and large-scale atmospheric effects (Ishaq & Jamel, 2026). The probability density function (PDF) of the irradiance fluctuations is given by:
The parameters
Where
Here,
For aperture averaging effect with aperture diameter D, the parameters are modified as:
Where
The geometric loss due to beam divergence is:
Where
In addition to beam divergence, bore sight pointing errors significantly impact the received power in FSO links, especially over long distances or when the optical alignment is unstable. These errors occur due to mechanical vibrations, wind-induced sway, or thermal drift, and result in a misalignment between the transmitter and receiver axes (Sharma et al., 2019). The received power with pointing error can be modeled as:
Where,
The atmospheric attenuation due to weather conditions follows Beer's Lambert law: (Maswikaneng et al., 2023)
Where
While Beer's law provides a deterministic estimate of attenuation based on average visibility, fog conditions in practice are random and spatially in homogeneous, especially over extended FSO links. To capture this, we model the atmospheric attenuation coefficient
Where
For dual-polarization systems, the Jones matrix representation of polarization effects is (Djordjevic, 2010):
Where
For an M-QAM modulation with constellation points
Where g(t) is the pulse shape and
For a square M-QAM constellation with
Where
The root-raised cosine (RRC) pulse shape is:
Where
The received signal after transmission through the FSO channel and addition of noise is:
The received signal is filtered with a matched filter and sampled at symbol rate: (Ghatwal & Saini, 2025)
For an M-QAM modulation, the frequency offset estimator based on 4th power method is:
The frequency correction is then:
For I/Q imbalance with gain imbalance
The LMS adaptive equalizer updates its coefficients according to:
For the RLS algorithm, the update equations are:
Where
For CMA (blind equalization), the update is:
Where R is the constant modulus.
The block-based CPE estimation for -QAM is:
The phase-corrected symbol is:
For -QAM in AWGN, the theoretical SER is approximately:
In a turbulent FSO channel with Gamma-Gamma distribution, the average SER is:
Where
For Gray-coded -QAM, the BER can be approximated as:
The ergodic capacity of the FSO channel with Gamma-Gamma turbulence is:
For dual polarization with polarization-dependent loss:
The EVM is defined as:
The adaptive modulation selects the modulation order based on the effective SNR:
The achievable data rate with adaptive modulation is:
The atmospheric attenuation coefficient
For fog and rain, empirical models are used: Light Fog: 1–5 dB/km Moderate Fog: 5–10 dB/km Heavy Fog: 10–20 dB/km Light Rain (5 mm/h): 1–2 dB/km Moderate Rain (12.5 mm/h): 2–5 dB/km Heavy Rain (25 mm/h): 5–10 dB/km
We adopted realistic parameters to emulate the most practical environment for free space optics. This simulation endorse our concept against dynamic nature of free space channel in large scale and small scale perturbations. Probability density function of irradiance fluctuations has been modeled in equation 1. Atmospheric conditions are modeled in depth using equations 2, 3, 4 by introducing rytov variance. Gamma gamma model for environmental factors due to scintillation has been employed. Fading effects modeled using equation 11. A range of environmental parameters from light fog to heavy rain has been used to check system endurance in a variety of environmental conditions which are hard to arrange in practical conditions. Log normal distribution has been used to model the stochastic fog. Besides realistic channel conditions, we have used the established models for components in transmitters and receivers. Starting from choosing wavelength of 1550 nm which is standard for telecommunications, aperture size, beam divergence parameters mentioned in Table 1 signify the compliance of our work with practical scenarios.
System Parameters and Specifications.
We simulate the dual-polarization FSO system using MATLAB/Python with the above components. The optical wavelength is 1550 nm (typical for FSO links due to eye safety). The link distance varies from 1 km up to 10 km to test performance. The transmitter aperture diameter is 20 cm and receiver aperture 40 cm, which helps collect more light and reduce diffraction losses (these sizes are feasible for building-mounted FSO terminals). We incorporate beam divergence (on the order of 0.01 mill radian) and pointing error with a small variance to simulate misalignment jitter.
In addition to fixed attenuation models, we also simulated random fog, where the fog-induced attenuation varies randomly along the link using a log-normal distribution with a standard deviation of 2–3 dB/km. This −provides a more realistic test of link stability under variable weather, reflecting microclimate changes across the FSO path. The results in Section 4.5 demonstrate that even under such stochastic fog, the system maintains acceptable BER using adaptive modulation and DSP.
Atmospheric turbulence is modeled using the Gamma-Gamma distribution for irradiance fluctuations, which is appropriate for a wide range of turbulence conditions (from weak to strong). We use refractive index structure parameter
Turbulence Condition Performance Comparison (16-QAM).
Turbulence Condition Performance Comparison (16-QAM).
Values are from Monte-Carlo simulation under the Gamma–Gamma turbulence model for irradiance (Al-Habash–Andrews–Phillips) with our Table 1 parameters and pointing-error statistics; methodology follows standard FSO analyses and yields BER/SNR trends consistent with published M-QAM results under turbulence.
Figure 2 compares the BER performance of the adaptive modulation scheme against a fixed-modulation baseline (fixed QPSK) under different turbulence strengths (weak, moderate, strong). The curves show BER as a function of SNR for each turbulence condition. Figure 2 indicates that under weak turbulence (nearly clear air), both adaptive and fixed QPSK achieve very low BER (below

Performance of adaptive vs. fixed modulation under different turbulence conditions.
At high SNR (e.g., >25 dB), the adaptive scheme in weak turbulence uses 256-QAM, which yields a slightly higher BER than QPSK (since 256-QAM is more sensitive to noise), but still below
Adaptive Modulation Performance (Full DSP Stack, 10 km).
Capacity gain is calculated as the percentage increase in effective capacity (bits/symbol × (1-BER)) compared to fixed QPSK modulation.
Figure 3 plots the channel capacity (in bits/symbol) achieved with adaptive modulation as a function of SNR for different turbulence conditions. This capacity is essentially the average number of bits per symbol used on each polarization (since our adaptation aims to maximize this while keeping BER low). As expected, higher SNR and weaker turbulence yield greater spectral efficiency. In clear air, the capacity rises to ∼8 bits/symbol at high SNR (meaning 256-QAM was reliably used). Under strong turbulence, the capacity curve grows more slowly with SNR – even at 30 dB SNR, the effective spectral efficiency is only ∼4–5 bits/symbol, because the fluctuations force the system to frequently downshift to QPSK or 16-QAM. Figure 3 highlights that at an SNR of 20 dB, the adaptive system in weak turbulence achieves about 6 bits/symbol (between 64-QAM and 256-QAM), whereas in strong turbulence it achieves only ∼2 bits/symbol (roughly QPSK). This quantifies the cost of turbulence in terms of lost spectral efficiency. Still, even in strong turbulence the adaptive link is utilizing the channel efficiently by adjusting the rate – a fixed 64-QAM system in those conditions would have failed (BER ∼

Channel capacity with adaptive modulation under different turbulence conditions.
Modulation Scheme Performance Comparison at 10 km Link Distance.
BER mentioned in Tables 2 and 4 are Raw BER before applying the impairments mitigation techniques under long reach conditions. It is acceptable until it is within the correction capacity of error correction schemes. Although these values of 10−1 and 10−2 appears higher for conventional systems, these parameters necessitate their implementation of adaptive modulation and deployment of DSP stack for error correction. We have described above in section 4.1 that adaptive modulation maintained the BER in range of 10-3 in case of severe fog conditions. It made signal quality in the range of application of FEC.
How the modulation changes with SNR and channel conditions, Figure 4 plots the modulation order (bits per symbol) chosen by the adaptive algorithm as a function of SNR, for the three turbulence levels. Each curve appears as a stepwise function – at low SNR, the system uses the lowest order (1 bit/sym, i.e., OOK/BPSK), then steps up to 2 bits/sym (QPSK) at some SNR threshold, then 4 bits (16-QAM), and so on. The thresholds occur at higher SNR for worse turbulence. For instance, in weak turbulence, the adaptive controller switches from QPSK to 16-QAM around 10 dB SNR, and from 16-QAM to 64-QAM around 20 dB. In strong turbulence, those transitions are delayed – QPSK persists until ∼18 dB, and 16-QAM until ∼28 dB, after which 64-QAM might barely be used. Figure 4 shows that under severe turbulence, the highest-order modulation (8 bits/sym, 256-QAM) is never reached even at 30 + dB SNR (the curve for strong turbulence stays at 6 bits/sym max), whereas in weak turbulence the system reaches 8 bits/sym at around 25 dB. This behavior reflects the controller ensuring a target BER (e.g., <

Adaptive modulation selection under different turbulence conditions.
Figure 5 translates the spectral efficiency into achievable data rate (Gbps) as a function of SNR under different turbulence conditions, assuming a fixed symbol rate and dual polarization. In our setup, the symbol rate is 10 GBd per polarization (so 20 Gbaud total across two polarization). Thus, 1 bit/sym corresponds to 20 Gbps, 8 bits/sym to 160 Gbps. We also mark a target line at 70 Gbps, representing a hypothetical required throughput for a 5G back haul link. We observe in Figure 5 that in weak turbulence, the dual-polarization FSO link can exceed 70 Gbps at modest SNR (∼10 dB) and reach a maximum of ∼150 Gbps at high SNR (using 256-QAM). Under moderate turbulence, the 70 Gbps rate is achieved at around 18 dB SNR. But under strong turbulence, even at 25 dB the adaptive link only delivers ∼60 Gbps – it fails to hit the 70 Gbps mark because the system must use lower modulations to maintain reliability. This suggests for very turbulence-prone links, either higher transmit power or some form of diversity would be needed to meet a 5G-level target consistently. Nonetheless, the adaptive system always outperforms a non-adaptive one in terms of throughput. For example, at 20 dB SNR and strong turbulence, the adaptive throughput is ∼40 Gbps, whereas a fixed QPSK system would carry only 40 Gbps at best (and likely less, since QPSK might be marginal) and a fixed 16-QAM system would likely be in outage (so effectively 0 Gbps reliable throughput). Thus, adaptation brings the advantage of graceful degradation: instead of an abrupt link failure when conditions worsen, the FSO link simply delivers a lower rate but maintains the connection.

Achievable data rate with adaptive modulation under different turbulence conditions.
Atmospheric attenuation due to weather drastically affects link SNR and thus performance. Figure 6 shows the BER vs SNR for three representative weather conditions: clear, light haze, and heavy fog, assuming full DSP is used. These curves were obtained by incorporating attenuation (e.g., 0, 5, and 15 dB/km, respectively) on top of moderate turbulence. As expected, additional attenuation shifts the BER curves to the right, effectively requiring higher SNR to achieve the same BER. In clear weather, a BER of

BER performance under different weather conditions with full DSP stack.
The inclusion of advanced DSP algorithms greatly improves link robustness. To demonstrate this, we tested the system's performance with and without the full DSP stack under a challenging scenario (moderate fog, moderate turbulence). Figure 7 compares BER vs SNR for two cases: (1) “No DSP” beyond basic filtering (meaning no adaptive equalizer or phase correction, equivalent to a simple direct-detection receiver), and (2) “Full DSP” including polarization/channel equalization and phase/timing recovery. The BER with full DSP is significantly lower at all SNRs. For instance, at 15 dB SNR, the no-DSP BER is around

BER performance of different DSP techniques under moderate fog.
DSP Technique Performance Comparison (16-QAM).
N represents the number of filters taps in the equalizer.
We also examined the Error Vector Magnitude (EVM) as a function of SNR for different DSP schemes. Figure 8 shows representative EVM results for moderate fog conditions comparing no DSP vs full DSP. At low SNR, EVM is high for both (dominated by noise), but as SNR increases, the full DSP case achieves much lower EVM. For example, at 20 dB SNR, the no-DSP EVM is ∼15%, whereas with full DSP it is ∼5%. Lower EVM directly correlates with lower symbol error rates and allows the use of higher-order modulation. In fact, in our adaptive modulation logic, we set an EVM threshold of ∼10% as the cutoff for switching to 64-QAM; the DSP reduced EVM sufficiently to enable 64-QAM at SNRs where it would otherwise be marginal. These results underscore that DSP extends the usable SNR range of each modulation format, translating to either improved throughput (since higher modulations can be used confidently) or improved receiver sensitivity (in terms of required SNR for a given BER).

EVM performance of different DSP techniques under moderate fog.
Figure 9 provides a bar-chart comparison of BER and capacity in different weather conditions at a fixed high SNR (25 dB) to summarize the weather and DSP effects. The left panel shows the log-BER for Clear, Haze, Fog under full DSP. Clear and haze both achieve BER well below

BER and capacity comparison at SNR = 24 dB.
Weather Condition Performance Comparison (16-QAM).
We specifically investigated how polarization-dependent effects impact the system and how well the DSP can mitigate them.
Figure 10 presents three subplots of BER versus (a) PDL, (b) polarization crosstalk, and (c) polarization rotation angle, measured at a fixed SNR (25 dB) with moderate turbulence. In the left subplot, we introduce an increasing polarization-dependent loss (PDL) between the two channels (0 to 3 dB imbalance) . Without compensation, PDL would mean one polarization has lower SNR than the other, potentially dominating the overall BER. We found that up to 1–2 dB PDL, the DSP equalizer can adjust gains to balance the channels, and BER stays around

Polarization effects at SNR = 25 dB with 16-QAM.
The middle subplot of Figure 10 examines polarization crosstalk, where we feed a fraction of each signal into the orthogonal polarization (e.g., due to non-ideal separation in the PBS or depolarization in the channel). We vary the crosstalk level from −30 dB (negligible) to −10 dB (significant leakage). The BER remains under
The right subplot of Figure 10 looks at polarization rotation misalignment. Here we rotate the polarization of the incoming signal relative to the receiver's reference axis (which could happen if the transmitter and receiver aren’t oriented the same). Even moderate rotation causes each received polarization to be a mix of the two transmitted ones. The BER vs rotation curve in Figure 10 shows very little penalty up to about 15°, and then a gradual increase. At 45° misalignment (meaning the polarization axes are completely crossed), BER is around
Polarization Effects Comparison (16-QAM, 25 dB SNR, 10 km).
In this article,we have extended the potential of free space systems by using dual polarization,higher order modulations,diverse environmental conditions from scintillation to fog and effective use of adaptive modulation and DSP algorithms for improving signal at the receiver.To best of our knowledge, this combination of parameters are in study of free space optics. Table 8 provides the brief comparison with some of resembling research articles.This highlights the novelty in our work.
Comparison with Systems in Other Reported Works.
We have presented a comprehensive performance analysis of a dual-polarization FSO communication system tailored for 5G and beyond networks. The system utilizes adaptive modulation and advanced DSP techniques to combat atmospheric turbulence and polarization-related impairments. Key findings from proposed system can be summarized as follows: Using orthogonal polarization doubles the link capacity, enabling data rates well over 100 Gbps on a single FSO link under good conditions. Even under moderate turbulence and attenuation, the dual-polarized system maintained significantly higher throughput than a single-polarization link, confirming the practical value of polarization multiplexing for 5G backhaul/front haul. Adaptive modulation proved crucial in maintaining low BER across a wide range of channel conditions. The system autonomously shifts between OOK, QPSK, 16-QAM, 64-QAM, and 256-QAM based on real-time channel quality estimates. This approach kept the BER below The incorporation of digital filtering, polarization de multiplexing, and adaptive equalization algorithms markedly improved link robustness. DSP reduced the required SNR by 3–5 dB for a target BER, effectively translating to increased link margin or distance coverage. It also corrected polarization misalignment up to ∼45° and crosstalk leaks up to ∼−15 dB, indicating that polarization multiplexing can be reliably realized with manageable alignment effort. Error vector magnitudes were substantially lowered by the DSP, yielding clean constellations and enabling higher-order modulations to perform error-free. The performance demonstrated shows that a single dual-polarization FSO link can handle the aggregated traffic of numerous 5G base stations or serve as a high-speed backbone in a front-haul network. The low latency and high security of the optical wireless channel make it suitable for mission-critical communications. The adaptive and robust nature of the link means network operators can count on a high level of availability (the link adapts to maintain a connection except in the most extreme weather, which occurs fractionally). This provides confidence in using FSO for carrier-grade services, with proper network planning for redundancy during rare outages.
In conclusion, our proposed system indicates that dual-polarization FSO technology, enhanced with adaptive modulation and sophisticated DSP, is a viable and potent solution for delivering fiber-like performance in 5G and future networks. It effectively addresses the traditional challenges of FSO (through real-time adaptation to turbulence and weather) and pushes the capacity to new heights, thereby widening the application scope of FSO links. Future networks beyond 5G, which may demand even higher data rates and more dynamic reconfiguration (for example, in a 6G ultra-dense network or a satellite-to-ground optical link scenario), will benefit from the techniques developed in this work. We have essentially shown a pathway to make FSO links smart – able to sense their environment and optimize accordingly – which is a key attribute of next-generation communication infrastructure.
Footnotes
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
